2024-09-13T17:28:32.015 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T17:28:32.034 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:28:32.134 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909 branch: wip-rishabh-testing-20240910.100730-reef description: fs/top/{begin/{0-install 1-ceph 2-logrotate 3-modules} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/fstop} email: null first_in_suite: false job_id: '7903909' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi meta: - desc: 1 ceph cluster with 1 mon, 1 mgr, 3 osds, 2 mds, 2 clients mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi no_nested_subset: false os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-rishabh-testing-20240910.100730-reef ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240910.100730-reef sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.b - osd.0 - osd.1 - osd.2 - client.0 - client.1 seed: 6439 sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 sleep_before_teardown: 0 subset: 964/1024 suite: fs suite_branch: wip-rishabh-testing-20240910.100730-reef suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH222ygDrwd7MpLaLNWW3/ck6qjAjg4Q2jbxdAMpTI4+At5hI0iIlfvaaaBDeDEszA0pRkfzuYP53TxfOnL9XNM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_fstop teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-13_14:42:16 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T17:28:32.134 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa; will attempt to use it 2024-09-13T17:28:32.135 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa/tasks 2024-09-13T17:28:32.135 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T17:28:32.136 INFO:teuthology.task.internal:Checking packages... 2024-09-13T17:28:32.157 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3' 2024-09-13T17:28:32.157 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T17:28:32.158 INFO:teuthology.packaging:ref: None 2024-09-13T17:28:32.158 INFO:teuthology.packaging:tag: None 2024-09-13T17:28:32.158 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:28:32.158 INFO:teuthology.packaging:sha1: 6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:28:32.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-rishabh-testing-20240910.100730-reef 2024-09-13T17:28:32.307 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-860-g6c6b083f-1jammy 2024-09-13T17:28:32.308 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T17:28:32.314 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T17:28:32.315 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T17:28:32.320 INFO:teuthology.task.internal:Saving configuration 2024-09-13T17:28:32.336 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T17:28:32.342 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T17:28:32.374 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 17:22:58.783812', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN6k11RAF34PdcpBqiYPSWf7ZZ1OkEcamoUwDeVMCYo1'} 2024-09-13T17:28:32.374 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T17:28:32.380 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1'] 2024-09-13T17:28:32.380 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T17:28:32.426 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff3e607bbe0>, signals=[15]) 2024-09-13T17:28:32.426 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T17:28:32.433 INFO:teuthology.task.internal:Opening connections... 2024-09-13T17:28:32.433 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:28:32.434 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:28:32.525 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T17:28:32.532 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-13T17:28:32.645 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-13T17:28:32.645 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="Ubuntu" 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="22.04" 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_CODENAME=jammy 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:ID=ubuntu 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE=debian 2024-09-13T17:28:32.691 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-13T17:28:32.692 INFO:teuthology.orchestra.run.smithi096.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-13T17:28:32.692 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-13T17:28:32.692 INFO:teuthology.orchestra.run.smithi096.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-13T17:28:32.692 INFO:teuthology.orchestra.run.smithi096.stdout:UBUNTU_CODENAME=jammy 2024-09-13T17:28:32.692 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-13T17:28:32.722 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T17:28:32.731 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T17:28:32.736 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T17:28:32.737 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-13T17:28:32.741 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T17:28:32.747 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T17:28:32.748 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-13T17:28:32.807 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T17:28:32.848 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T17:28:32.862 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T17:28:32.862 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T17:28:32.862 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T17:28:32.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:28:32.867 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-13T17:28:32.916 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-56-generic 2024-09-13T17:28:32.916 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.15.0-56-generic 2024-09-13T17:28:32.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-13T17:28:33.068 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-13T17:28:33.208 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T17:28:33.208 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-13T17:28:33.208 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-13T17:28:33.280 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-13T17:28:33.452 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,806 kB] 2024-09-13T17:28:33.522 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,023 kB] 2024-09-13T17:28:33.556 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [539 kB] 2024-09-13T17:28:33.560 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [295 kB] 2024-09-13T17:28:33.562 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-13T17:28:33.562 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-13T17:28:33.563 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,377 kB] 2024-09-13T17:28:33.583 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [409 kB] 2024-09-13T17:28:33.588 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-13T17:28:33.588 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [902 kB] 2024-09-13T17:28:33.595 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [627 kB] 2024-09-13T17:28:33.599 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-13T17:28:33.600 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-13T17:28:33.600 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-13T17:28:33.601 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-13T17:28:33.625 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [696 kB] 2024-09-13T17:28:33.625 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-13T17:28:33.625 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-13T17:28:33.630 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [352 kB] 2024-09-13T17:28:33.632 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-13T17:28:33.632 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,437 kB] 2024-09-13T17:28:33.650 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-13T17:28:33.651 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [419 kB] 2024-09-13T17:28:33.654 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-13T17:28:33.654 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-13T17:28:33.661 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-13T17:28:33.668 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-13T17:28:33.670 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-13T17:28:33.671 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-13T17:28:33.671 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-13T17:28:33.682 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-13T17:28:33.682 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-13T17:28:33.683 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-13T17:28:33.683 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-13T17:28:33.684 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-13T17:28:33.699 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-13T17:28:33.699 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-13T17:28:33.700 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-13T17:28:33.700 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-13T17:28:33.700 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-13T17:28:39.908 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 16.0 MB in 2s (7,420 kB/s) 2024-09-13T17:28:41.507 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:28:41.527 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:28:41.590 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:28:41.799 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-13T17:28:41.800 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-13T17:28:41.980 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:28:41.980 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T17:28:41.980 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T17:28:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T17:28:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-13T17:28:41.981 INFO:teuthology.orchestra.run.smithi096.stdout: python2.7-minimal 2024-09-13T17:28:41.981 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:28:41.982 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-13T17:28:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T17:28:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-generic linux-image-5.15.0-121-generic 2024-09-13T17:28:41.982 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-5.15.0-121-generic linux-modules-extra-5.15.0-121-generic 2024-09-13T17:28:41.984 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-13T17:28:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-13T17:28:41.984 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-13T17:28:41.984 INFO:teuthology.orchestra.run.smithi096.stdout: thermald 2024-09-13T17:28:42.034 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-13T17:28:42.034 INFO:teuthology.orchestra.run.smithi096.stdout: linux-headers-5.15.0-121 linux-headers-5.15.0-121-generic 2024-09-13T17:28:42.034 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.15.0-121-generic linux-modules-5.15.0-121-generic 2024-09-13T17:28:42.035 INFO:teuthology.orchestra.run.smithi096.stdout: linux-modules-extra-5.15.0-121-generic 2024-09-13T17:28:42.036 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-13T17:28:42.036 INFO:teuthology.orchestra.run.smithi096.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-13T17:28:42.217 INFO:teuthology.orchestra.run.smithi096.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T17:28:42.217 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 113 MB of archives. 2024-09-13T17:28:42.217 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-13T17:28:42.218 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-121-generic amd64 5.15.0-121.131 [22.7 MB] 2024-09-13T17:28:43.594 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-121-generic amd64 5.15.0-121.131 [11.5 MB] 2024-09-13T17:28:43.950 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-121-generic amd64 5.15.0-121.131 [63.9 MB] 2024-09-13T17:28:46.138 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.121.121 [1,694 B] 2024-09-13T17:28:46.138 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.121.121 [2,464 B] 2024-09-13T17:28:46.138 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121 all 5.15.0-121.131 [12.3 MB] 2024-09-13T17:28:46.488 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-121-generic amd64 5.15.0-121.131 [2,827 kB] 2024-09-13T17:28:46.572 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.121.121 [2,316 B] 2024-09-13T17:28:46.976 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 113 MB in 5s (25.0 MB/s) 2024-09-13T17:28:47.202 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-5.15.0-121-generic. 2024-09-13T17:28:49.218 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-13T17:28:49.222 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../0-linux-modules-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T17:28:49.325 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:28:51.986 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-image-5.15.0-121-generic. 2024-09-13T17:28:52.007 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../1-linux-image-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T17:28:52.096 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:28:52.540 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-121-generic. 2024-09-13T17:28:52.560 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T17:28:52.597 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:28:59.142 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../3-linux-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T17:28:59.260 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T17:28:59.605 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T17:28:59.714 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-image-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T17:29:00.026 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-121. 2024-09-13T17:29:00.037 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../5-linux-headers-5.15.0-121_5.15.0-121.131_all.deb ... 2024-09-13T17:29:00.076 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T17:29:06.152 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package linux-headers-5.15.0-121-generic. 2024-09-13T17:29:06.188 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../6-linux-headers-5.15.0-121-generic_5.15.0-121.131_amd64.deb ... 2024-09-13T17:29:06.226 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:09.762 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.121.121_amd64.deb ... 2024-09-13T17:29:09.887 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking linux-headers-generic (5.15.0.121.121) over (5.15.0.56.54) ... 2024-09-13T17:29:10.229 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-121 (5.15.0-121.131) ... 2024-09-13T17:29:10.350 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:10.494 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-headers-generic (5.15.0.121.121) ... 2024-09-13T17:29:10.621 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:11.949 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-13T17:29:11.949 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-13T17:29:11.949 INFO:teuthology.orchestra.run.smithi096.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-121-generic 2024-09-13T17:29:11.949 INFO:teuthology.orchestra.run.smithi096.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-121-generic 2024-09-13T17:29:12.090 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:13.833 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-modules-extra-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:15.784 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-image-generic (5.15.0.121.121) ... 2024-09-13T17:29:15.935 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up linux-generic (5.15.0.121.121) ... 2024-09-13T17:29:16.053 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for linux-image-5.15.0-121-generic (5.15.0-121.131) ... 2024-09-13T17:29:16.110 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-13T17:29:16.111 INFO:teuthology.orchestra.run.smithi096.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-121-generic 2024-09-13T17:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-13T17:29:30.279 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub' 2024-09-13T17:29:30.292 INFO:teuthology.orchestra.run.smithi096.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:29:30.317 INFO:teuthology.orchestra.run.smithi096.stdout:Generating grub configuration file ... 2024-09-13T17:29:30.698 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T17:29:30.715 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T17:29:31.154 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T17:29:31.159 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T17:29:31.234 INFO:teuthology.orchestra.run.smithi096.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T17:29:31.239 INFO:teuthology.orchestra.run.smithi096.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T17:29:31.395 INFO:teuthology.orchestra.run.smithi096.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T17:29:31.395 INFO:teuthology.orchestra.run.smithi096.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T17:29:31.395 INFO:teuthology.orchestra.run.smithi096.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T17:29:31.437 INFO:teuthology.orchestra.run.smithi096.stdout:done 2024-09-13T17:29:31.920 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-13T17:29:31.945 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.121.121 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:29:31.946 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-13T17:29:31.946 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-121-generic 2024-09-13T17:29:31.947 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T17:29:31.947 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T17:29:31.947 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T17:29:31.947 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi096.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T17:29:31.948 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get clean 2024-09-13T17:29:32.089 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-13T17:29:32.211 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T17:29:32.334 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T17:29:32.416 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T17:29:32.498 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T17:29:33.635 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:29:33.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-13T17:29:33.707 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:29:33.908 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-13T17:29:33.908 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-13T17:29:34.075 INFO:teuthology.orchestra.run.smithi096.stdout:linux-image-generic is already the newest version (5.15.0.121.121). 2024-09-13T17:29:34.075 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:29:34.075 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T17:29:34.075 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T17:29:34.075 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T17:29:34.076 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T17:29:34.076 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T17:29:34.076 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:29:34.116 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-13T17:29:34.118 DEBUG:teuthology.orchestra.run.smithi096:> dpkg -s linux-image-generic 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Package: linux-image-generic 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Status: install ok installed 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Priority: optional 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Section: kernel 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Installed-Size: 21 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Maintainer: Ubuntu Kernel Team 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Architecture: amd64 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Source: linux-meta 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Version: 5.15.0.121.121 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-121), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Depends: linux-image-5.15.0-121-generic, linux-modules-extra-5.15.0-121-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-13T17:29:34.177 INFO:teuthology.orchestra.run.smithi096.stdout:Recommends: thermald 2024-09-13T17:29:34.178 INFO:teuthology.orchestra.run.smithi096.stdout:Description: Generic Linux kernel image 2024-09-13T17:29:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: This package will always depend on the latest generic kernel image 2024-09-13T17:29:34.178 INFO:teuthology.orchestra.run.smithi096.stdout: available. 2024-09-13T17:29:34.178 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-13T17:29:34.222 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.5nw5iguVTL 2024-09-13T17:29:34.222 DEBUG:teuthology.orchestra.run.smithi096:> sudo cp /boot/grub/grub.cfg /tmp/tmp.5nw5iguVTL 2024-09-13T17:29:34.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.5nw5iguVTL 2024-09-13T17:29:34.390 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.5nw5iguVTL is 10KB 2024-09-13T17:29:34.400 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.5nw5iguVTL 2024-09-13T17:29:34.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-13T17:29:34.458 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:29:34.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-13T17:29:34.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-13T17:29:34.526 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-121-generic 2024-09-13T17:29:34.526 DEBUG:teuthology.orchestra.run.smithi096:> sudo update-grub 2024-09-13T17:29:35.826 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub' 2024-09-13T17:29:35.827 INFO:teuthology.orchestra.run.smithi096.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-13T17:29:35.830 INFO:teuthology.orchestra.run.smithi096.stderr:Generating grub configuration file ... 2024-09-13T17:29:36.112 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-121-generic 2024-09-13T17:29:36.118 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-121-generic 2024-09-13T17:29:36.424 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-13T17:29:36.430 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-13T17:29:36.474 INFO:teuthology.orchestra.run.smithi096.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-13T17:29:36.479 INFO:teuthology.orchestra.run.smithi096.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-13T17:29:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-13T17:29:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-13T17:29:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-13T17:29:36.610 INFO:teuthology.orchestra.run.smithi096.stderr:done 2024-09-13T17:29:36.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo shutdown -r now 2024-09-13T17:30:06.685 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T17:30:06.686 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:30:06.687 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:30:41.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-13T17:30:50.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:30:50.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:31:08.733 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.96 2024-09-13T17:31:20.745 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:31:20.746 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:31:21.148 ERROR:teuthology.orchestra.connection:Error authenticating with smithi096.front.sepia.ceph.com: SSHException('No existing session') (No SSH private key found!) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/connection.py", line 111, in connect ssh.connect(**connect_args) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 485, in connect self._auth( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 818, in _auth raise saved_exception File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/client.py", line 794, in _auth self._transport.auth_publickey(username, key) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/virtualenv/lib/python3.10/site-packages/paramiko/transport.py", line 1664, in auth_publickey raise SSHException("No existing session") paramiko.ssh_exception.SSHException: No existing session 2024-09-13T17:31:25.241 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-13T17:31:26.424 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:31:26.424 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-121-generic"... 2024-09-13T17:31:26.424 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-09-13T17:31:26.472 INFO:teuthology.orchestra.run.smithi096.stdout:5.15.0-121-generic 2024-09-13T17:31:26.472 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-121-generic vs 5.15.0-121-generic 2024-09-13T17:31:26.472 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T17:31:26.473 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-13T17:31:27.473 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T17:31:27.473 DEBUG:teuthology.orchestra.run.smithi096:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T17:31:27.556 INFO:teuthology.orchestra.run.smithi096.stdout:ttyS1 2024-09-13T17:31:27.578 DEBUG:teuthology.parallel:result is None 2024-09-13T17:31:27.578 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T17:31:27.585 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T17:31:27.585 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T17:31:27.591 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T17:31:27.597 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T17:31:27.607 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T17:31:27.608 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T17:31:27.652 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T17:31:27.659 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T17:31:27.660 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T17:31:27.711 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:31:27.719 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T17:31:27.720 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T17:31:27.727 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T17:31:27.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T17:31:27.786 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T17:31:27.796 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T17:31:27.796 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T17:31:27.833 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T17:31:27.885 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T17:31:27.933 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:31:27.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T17:31:28.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-13T17:31:28.086 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T17:31:28.094 INFO:teuthology.task.internal:Starting timer... 2024-09-13T17:31:28.094 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T17:31:28.108 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T17:31:28.120 DEBUG:teuthology.task.selinux:Excluding smithi096: OS 'ubuntu' does not support SELinux 2024-09-13T17:31:28.120 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T17:31:28.120 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-13T17:31:28.120 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T17:31:28.120 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T17:31:28.131 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-13T17:31:28.133 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-13T17:31:28.295 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T17:31:28.305 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T17:31:28.306 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T17:38:55.160 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com')] 2024-09-13T17:38:55.161 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:38:55.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T17:38:55.257 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-13T17:38:55.337 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-13T17:38:55.338 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T17:38:55.348 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T17:38:55.348 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T17:38:55.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:38:55.408 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-13T17:38:55.408 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Command line: ntpd -gq 2024-09-13T17:38:55.408 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: ---------------------------------------------------- 2024-09-13T17:38:55.408 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: ntp-4 is maintained by Network Time Foundation, 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: corporation. Support and training for ntp-4 are 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: available at https://www.nwtime.org/support 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: ---------------------------------------------------- 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: proto: precision = 0.061 usec (-24) 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: basedate set to 2022-02-04 2024-09-13T17:38:55.409 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: gps base set to 2022-02-06 (week 2196) 2024-09-13T17:38:55.410 INFO:teuthology.orchestra.run.smithi096.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:38:55.410 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-13T17:38:55.410 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:38:55.410 INFO:teuthology.orchestra.run.smithi096.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-13T17:38:55.411 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen and drop on 0 v6wildcard [::]:123 2024-09-13T17:38:55.411 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-13T17:38:55.412 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen normally on 2 lo 127.0.0.1:123 2024-09-13T17:38:55.412 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen normally on 3 enp3s0f1 172.21.15.96:123 2024-09-13T17:38:55.412 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen normally on 4 lo [::1]:123 2024-09-13T17:38:55.412 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cc0d%5]:123 2024-09-13T17:38:55.412 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:38:55 ntpd[18489]: Listening on routing socket on fd #22 for interface updates 2024-09-13T17:38:56.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:56.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:56.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:56.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:56.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:38:56.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:56 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:38:57.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:57 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:57.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:57 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:58.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:38:58.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:38:58 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:39:00.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:00.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:39:00.412 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:00 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:39:02.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:02.409 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/rawstats.20240913: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/peerstats.20240913: Permission denied 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stdout:13 Sep 17:39:02 ntpd[18489]: ntpd: time slew -0.000827 s 2024-09-13T17:39:02.410 INFO:teuthology.orchestra.run.smithi096.stdout:ntpd: time slew -0.000827s 2024-09-13T17:39:02.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-13T17:39:02.411 INFO:teuthology.orchestra.run.smithi096.stderr:13 Sep 17:39:02 ntpd[18489]: can't open /var/log/ntpstats/loopstats.20240913: Permission denied 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T17:39:02.452 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T17:39:02.452 INFO:teuthology.run_tasks:Running task install... 2024-09-13T17:39:02.460 DEBUG:teuthology.task.install:project ceph 2024-09-13T17:39:02.460 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}} 2024-09-13T17:39:02.460 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'} 2024-09-13T17:39:02.460 INFO:teuthology.task.install:Using flavor: default 2024-09-13T17:39:02.466 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T17:39:02.467 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-09-13T17:39:02.467 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-key list | grep Ceph 2024-09-13T17:39:02.547 INFO:teuthology.orchestra.run.smithi096.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-13T17:39:02.573 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-13T17:39:02.573 INFO:teuthology.orchestra.run.smithi096.stdout:uid [ unknown] Ceph.com (release key) 2024-09-13T17:39:02.574 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-13T17:39:02.574 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-13T17:39:02.574 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:39:02.728 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default/ 2024-09-13T17:39:02.728 INFO:teuthology.task.install.deb:Package version is 18.2.4-860-g6c6b083f-1jammy 2024-09-13T17:39:02.852 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:39:02.852 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-13T17:39:02.866 DEBUG:teuthology.orchestra.run.smithi096:> sudo apt-get update 2024-09-13T17:39:03.089 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy InRelease 2024-09-13T17:39:03.111 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-13T17:39:03.135 INFO:teuthology.orchestra.run.smithi096.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-13T17:39:03.157 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.7 kB] 2024-09-13T17:39:03.181 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-13T17:39:03.181 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-13T17:39:03.189 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,428 B] 2024-09-13T17:39:03.255 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-13T17:39:03.337 INFO:teuthology.orchestra.run.smithi096.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-13T17:39:03.549 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 28.8 kB in 1s (57.5 kB/s) 2024-09-13T17:39:04.586 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:39:04.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.4-860-g6c6b083f-1jammy cephadm=18.2.4-860-g6c6b083f-1jammy ceph-mds=18.2.4-860-g6c6b083f-1jammy ceph-mgr=18.2.4-860-g6c6b083f-1jammy ceph-common=18.2.4-860-g6c6b083f-1jammy ceph-fuse=18.2.4-860-g6c6b083f-1jammy ceph-test=18.2.4-860-g6c6b083f-1jammy ceph-volume=18.2.4-860-g6c6b083f-1jammy radosgw=18.2.4-860-g6c6b083f-1jammy python3-rados=18.2.4-860-g6c6b083f-1jammy python3-rgw=18.2.4-860-g6c6b083f-1jammy python3-cephfs=18.2.4-860-g6c6b083f-1jammy python3-rbd=18.2.4-860-g6c6b083f-1jammy libcephfs2=18.2.4-860-g6c6b083f-1jammy libcephfs-dev=18.2.4-860-g6c6b083f-1jammy librados2=18.2.4-860-g6c6b083f-1jammy librbd1=18.2.4-860-g6c6b083f-1jammy rbd-fuse=18.2.4-860-g6c6b083f-1jammy python3-cephfs=18.2.4-860-g6c6b083f-1jammy cephfs-shell=18.2.4-860-g6c6b083f-1jammy cephfs-top=18.2.4-860-g6c6b083f-1jammy cephfs-mirror=18.2.4-860-g6c6b083f-1jammy 2024-09-13T17:39:04.667 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:39:04.876 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-13T17:39:04.876 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-13T17:39:05.067 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:39:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T17:39:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T17:39:05.067 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T17:39:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T17:39:05.068 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T17:39:05.068 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-13T17:39:05.069 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-13T17:39:05.070 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: qttranslations5-l10n socat xmlstarlet 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2024-09-13T17:39:05.071 INFO:teuthology.orchestra.run.smithi096.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2024-09-13T17:39:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2024-09-13T17:39:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: python3-watchdog 2024-09-13T17:39:05.072 INFO:teuthology.orchestra.run.smithi096.stdout:Recommended packages: 2024-09-13T17:39:05.072 INFO:teuthology.orchestra.run.smithi096.stdout: btrfs-tools 2024-09-13T17:39:05.178 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-13T17:39:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-13T17:39:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-13T17:39:05.178 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-13T17:39:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-13T17:39:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2024-09-13T17:39:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-13T17:39:05.179 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-13T17:39:05.180 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-13T17:39:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: socat xmlstarlet 2024-09-13T17:39:05.241 INFO:teuthology.orchestra.run.smithi096.stdout:0 upgraded, 115 newly installed, 0 to remove and 331 not upgraded. 2024-09-13T17:39:05.242 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 163 MB of archives. 2024-09-13T17:39:05.242 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 748 MB of additional disk space will be used. 2024-09-13T17:39:05.242 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-13T17:39:05.259 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-13T17:39:05.268 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-13T17:39:05.310 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2024-09-13T17:39:05.321 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-13T17:39:05.334 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.4-860-g6c6b083f-1jammy [3,453 kB] 2024-09-13T17:39:05.342 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-13T17:39:05.345 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-13T17:39:05.356 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-13T17:39:05.411 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-13T17:39:05.418 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-13T17:39:05.428 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-13T17:39:05.432 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-13T17:39:05.433 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-13T17:39:05.442 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-13T17:39:05.443 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-13T17:39:05.443 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-13T17:39:05.444 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-13T17:39:05.444 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-13T17:39:05.450 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-13T17:39:05.450 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-13T17:39:05.467 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-13T17:39:05.467 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-13T17:39:05.467 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-13T17:39:05.470 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-13T17:39:05.471 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-13T17:39:05.471 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-13T17:39:05.472 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-13T17:39:05.473 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-13T17:39:05.484 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-13T17:39:05.484 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-13T17:39:05.501 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-13T17:39:05.502 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-13T17:39:05.502 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-13T17:39:05.508 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-13T17:39:05.508 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-13T17:39:05.508 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-13T17:39:05.509 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-13T17:39:05.511 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-13T17:39:05.518 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-13T17:39:05.519 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-13T17:39:05.525 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.4-860-g6c6b083f-1jammy [3,108 kB] 2024-09-13T17:39:05.530 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-13T17:39:05.531 INFO:teuthology.orchestra.run.smithi096.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-13T17:39:05.533 INFO:teuthology.orchestra.run.smithi096.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-13T17:39:05.536 INFO:teuthology.orchestra.run.smithi096.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-13T17:39:05.537 INFO:teuthology.orchestra.run.smithi096.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-13T17:39:05.537 INFO:teuthology.orchestra.run.smithi096.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-13T17:39:05.571 INFO:teuthology.orchestra.run.smithi096.stdout:Get:49 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.4-860-g6c6b083f-1jammy [684 kB] 2024-09-13T17:39:05.582 INFO:teuthology.orchestra.run.smithi096.stdout:Get:50 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.4-860-g6c6b083f-1jammy [357 kB] 2024-09-13T17:39:05.586 INFO:teuthology.orchestra.run.smithi096.stdout:Get:51 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.4-860-g6c6b083f-1jammy [32.8 kB] 2024-09-13T17:39:05.587 INFO:teuthology.orchestra.run.smithi096.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.4-860-g6c6b083f-1jammy [180 kB] 2024-09-13T17:39:05.589 INFO:teuthology.orchestra.run.smithi096.stdout:Get:53 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.4-860-g6c6b083f-1jammy [63.7 kB] 2024-09-13T17:39:05.590 INFO:teuthology.orchestra.run.smithi096.stdout:Get:54 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.4-860-g6c6b083f-1jammy [334 kB] 2024-09-13T17:39:05.592 INFO:teuthology.orchestra.run.smithi096.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-13T17:39:05.593 INFO:teuthology.orchestra.run.smithi096.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-13T17:39:05.637 INFO:teuthology.orchestra.run.smithi096.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-13T17:39:05.638 INFO:teuthology.orchestra.run.smithi096.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.4-860-g6c6b083f-1jammy [5,933 kB] 2024-09-13T17:39:05.638 INFO:teuthology.orchestra.run.smithi096.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-13T17:39:05.638 INFO:teuthology.orchestra.run.smithi096.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-13T17:39:05.639 INFO:teuthology.orchestra.run.smithi096.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-13T17:39:05.639 INFO:teuthology.orchestra.run.smithi096.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-13T17:39:05.641 INFO:teuthology.orchestra.run.smithi096.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-13T17:39:05.641 INFO:teuthology.orchestra.run.smithi096.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-13T17:39:05.642 INFO:teuthology.orchestra.run.smithi096.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-13T17:39:05.660 INFO:teuthology.orchestra.run.smithi096.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-13T17:39:05.662 INFO:teuthology.orchestra.run.smithi096.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-13T17:39:05.662 INFO:teuthology.orchestra.run.smithi096.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-13T17:39:05.679 INFO:teuthology.orchestra.run.smithi096.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-13T17:39:05.679 INFO:teuthology.orchestra.run.smithi096.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-13T17:39:05.679 INFO:teuthology.orchestra.run.smithi096.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-13T17:39:05.680 INFO:teuthology.orchestra.run.smithi096.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-13T17:39:05.680 INFO:teuthology.orchestra.run.smithi096.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-13T17:39:05.681 INFO:teuthology.orchestra.run.smithi096.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-13T17:39:05.691 INFO:teuthology.orchestra.run.smithi096.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-13T17:39:05.692 INFO:teuthology.orchestra.run.smithi096.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-13T17:39:05.695 INFO:teuthology.orchestra.run.smithi096.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-13T17:39:05.704 INFO:teuthology.orchestra.run.smithi096.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.4-860-g6c6b083f-1jammy [112 kB] 2024-09-13T17:39:05.705 INFO:teuthology.orchestra.run.smithi096.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.4-860-g6c6b083f-1jammy [445 kB] 2024-09-13T17:39:05.708 INFO:teuthology.orchestra.run.smithi096.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-13T17:39:05.710 INFO:teuthology.orchestra.run.smithi096.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-13T17:39:05.710 INFO:teuthology.orchestra.run.smithi096.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-13T17:39:05.713 INFO:teuthology.orchestra.run.smithi096.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.4-860-g6c6b083f-1jammy [22.1 MB] 2024-09-13T17:39:05.714 INFO:teuthology.orchestra.run.smithi096.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-13T17:39:05.717 INFO:teuthology.orchestra.run.smithi096.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-13T17:39:05.717 INFO:teuthology.orchestra.run.smithi096.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-13T17:39:05.721 INFO:teuthology.orchestra.run.smithi096.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-13T17:39:05.725 INFO:teuthology.orchestra.run.smithi096.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-13T17:39:05.726 INFO:teuthology.orchestra.run.smithi096.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-13T17:39:05.727 INFO:teuthology.orchestra.run.smithi096.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-13T17:39:05.743 INFO:teuthology.orchestra.run.smithi096.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-13T17:39:05.743 INFO:teuthology.orchestra.run.smithi096.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-13T17:39:05.743 INFO:teuthology.orchestra.run.smithi096.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-13T17:39:06.199 INFO:teuthology.orchestra.run.smithi096.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.4-860-g6c6b083f-1jammy [5,006 kB] 2024-09-13T17:39:06.261 INFO:teuthology.orchestra.run.smithi096.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.4-860-g6c6b083f-1jammy [242 kB] 2024-09-13T17:39:06.264 INFO:teuthology.orchestra.run.smithi096.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.4-860-g6c6b083f-1jammy [130 kB] 2024-09-13T17:39:06.265 INFO:teuthology.orchestra.run.smithi096.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.4-860-g6c6b083f-1jammy [1,496 kB] 2024-09-13T17:39:06.278 INFO:teuthology.orchestra.run.smithi096.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.4-860-g6c6b083f-1jammy [6,085 kB] 2024-09-13T17:39:06.365 INFO:teuthology.orchestra.run.smithi096.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.4-860-g6c6b083f-1jammy [22.4 MB] 2024-09-13T17:39:06.816 INFO:teuthology.orchestra.run.smithi096.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.4-860-g6c6b083f-1jammy [14.2 kB] 2024-09-13T17:39:06.817 INFO:teuthology.orchestra.run.smithi096.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.4-860-g6c6b083f-1jammy [835 kB] 2024-09-13T17:39:06.826 INFO:teuthology.orchestra.run.smithi096.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.4-860-g6c6b083f-1jammy [2,159 kB] 2024-09-13T17:39:06.850 INFO:teuthology.orchestra.run.smithi096.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.4-860-g6c6b083f-1jammy [240 kB] 2024-09-13T17:39:06.853 INFO:teuthology.orchestra.run.smithi096.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.4-860-g6c6b083f-1jammy [145 kB] 2024-09-13T17:39:06.854 INFO:teuthology.orchestra.run.smithi096.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.4-860-g6c6b083f-1jammy [2,233 kB] 2024-09-13T17:39:06.877 INFO:teuthology.orchestra.run.smithi096.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.4-860-g6c6b083f-1jammy [8,625 kB] 2024-09-13T17:39:07.005 INFO:teuthology.orchestra.run.smithi096.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.4-860-g6c6b083f-1jammy [14.3 kB] 2024-09-13T17:39:07.005 INFO:teuthology.orchestra.run.smithi096.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.4-860-g6c6b083f-1jammy [42.9 MB] 2024-09-13T17:39:07.923 INFO:teuthology.orchestra.run.smithi096.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.4-860-g6c6b083f-1jammy [122 kB] 2024-09-13T17:39:07.925 INFO:teuthology.orchestra.run.smithi096.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.4-860-g6c6b083f-1jammy [233 kB] 2024-09-13T17:39:07.929 INFO:teuthology.orchestra.run.smithi096.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.4-860-g6c6b083f-1jammy [34.0 kB] 2024-09-13T17:39:07.930 INFO:teuthology.orchestra.run.smithi096.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.4-860-g6c6b083f-1jammy [23.5 kB] 2024-09-13T17:39:07.931 INFO:teuthology.orchestra.run.smithi096.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.4-860-g6c6b083f-1jammy [38.3 kB] 2024-09-13T17:39:07.932 INFO:teuthology.orchestra.run.smithi096.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.4-860-g6c6b083f-1jammy [11.7 MB] 2024-09-13T17:39:08.088 INFO:teuthology.orchestra.run.smithi096.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240910.100730-reef/6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.4-860-g6c6b083f-1jammy [93.0 kB] 2024-09-13T17:39:08.654 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 163 MB in 3s (56.3 MB/s) 2024-09-13T17:39:08.791 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-13T17:39:08.887 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-13T17:39:08.892 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-13T17:39:08.931 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-13T17:39:09.217 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-13T17:39:09.243 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-13T17:39:09.283 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T17:39:09.553 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cmd2. 2024-09-13T17:39:09.568 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-13T17:39:09.603 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-13T17:39:09.880 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfmt8:amd64. 2024-09-13T17:39:09.896 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2024-09-13T17:39:09.964 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T17:39:10.300 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-13T17:39:10.326 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-13T17:39:10.367 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T17:39:10.704 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-13T17:39:10.730 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-13T17:39:10.770 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T17:39:11.106 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-13T17:39:11.132 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-13T17:39:11.173 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T17:39:11.518 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-13T17:39:11.544 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T17:39:11.593 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:39:11.988 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-13T17:39:12.014 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T17:39:12.054 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:39:12.382 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-13T17:39:12.408 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-13T17:39:12.448 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:39:12.727 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-13T17:39:12.742 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-13T17:39:12.776 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T17:39:13.112 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librados2. 2024-09-13T17:39:13.139 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../011-librados2_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:13.179 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librados2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:13.632 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librbd1. 2024-09-13T17:39:13.658 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../012-librbd1_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:13.698 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librbd1 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:14.161 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs2. 2024-09-13T17:39:14.186 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../013-libcephfs2_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:14.226 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:14.487 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rados. 2024-09-13T17:39:14.502 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../014-python3-rados_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:14.537 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rados (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:14.823 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-13T17:39:14.849 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:14.889 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-argparse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:15.142 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cephfs. 2024-09-13T17:39:15.157 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../016-python3-cephfs_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:15.200 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cephfs (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:15.511 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-13T17:39:15.537 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:15.577 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-ceph-common (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:15.897 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-prettytable. 2024-09-13T17:39:15.912 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-13T17:39:15.947 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-13T17:39:16.166 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rbd. 2024-09-13T17:39:16.182 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../019-python3-rbd_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:16.216 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rbd (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:16.561 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-13T17:39:16.587 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-13T17:39:16.627 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T17:39:16.955 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package librgw2. 2024-09-13T17:39:16.981 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../021-librgw2_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:17.022 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking librgw2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:17.516 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rgw. 2024-09-13T17:39:17.531 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../022-python3-rgw_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:17.565 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rgw (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:17.877 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-13T17:39:17.903 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-13T17:39:17.944 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T17:39:18.263 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libradosstriper1. 2024-09-13T17:39:18.289 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../024-libradosstriper1_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:18.329 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libradosstriper1 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:18.758 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-common. 2024-09-13T17:39:18.784 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../025-ceph-common_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:18.824 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-common (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:20.080 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-base. 2024-09-13T17:39:20.106 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../026-ceph-base_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:20.155 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-base (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:20.717 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-13T17:39:20.743 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-13T17:39:20.786 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-13T17:39:21.072 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cheroot. 2024-09-13T17:39:21.098 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-13T17:39:21.138 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T17:39:21.425 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-13T17:39:21.440 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-13T17:39:21.475 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-13T17:39:21.785 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-13T17:39:21.812 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-13T17:39:21.852 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-13T17:39:22.155 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-13T17:39:22.181 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-13T17:39:22.221 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-13T17:39:22.516 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempora. 2024-09-13T17:39:22.545 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2024-09-13T17:39:22.582 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-13T17:39:22.835 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-portend. 2024-09-13T17:39:22.861 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2024-09-13T17:39:22.902 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-13T17:39:23.188 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-13T17:39:23.214 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-13T17:39:23.254 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-13T17:39:23.701 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-13T17:39:23.728 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-13T17:39:23.874 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-13T17:39:24.219 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-natsort. 2024-09-13T17:39:24.245 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2024-09-13T17:39:24.286 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-13T17:39:24.597 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-logutils. 2024-09-13T17:39:24.623 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2024-09-13T17:39:24.664 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-13T17:39:24.966 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-13T17:39:24.994 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-13T17:39:25.033 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-13T17:39:25.285 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-mako. 2024-09-13T17:39:25.301 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-13T17:39:25.377 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T17:39:25.706 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-13T17:39:25.732 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-13T17:39:25.772 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:39:26.075 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-13T17:39:26.101 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-13T17:39:26.141 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-13T17:39:26.469 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webob. 2024-09-13T17:39:26.495 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-13T17:39:26.544 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T17:39:26.855 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-waitress. 2024-09-13T17:39:26.881 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-13T17:39:26.925 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T17:39:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-tempita. 2024-09-13T17:39:27.209 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-13T17:39:27.249 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T17:39:27.535 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-paste. 2024-09-13T17:39:27.562 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-13T17:39:27.602 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T17:39:27.897 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-13T17:39:27.912 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-13T17:39:27.947 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T17:39:28.249 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-13T17:39:28.276 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-13T17:39:28.316 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-13T17:39:28.619 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-webtest. 2024-09-13T17:39:28.645 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2024-09-13T17:39:28.685 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-13T17:39:29.039 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pecan. 2024-09-13T17:39:29.065 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-13T17:39:29.105 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T17:39:29.468 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-certifi. 2024-09-13T17:39:29.494 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2024-09-13T17:39:29.535 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-13T17:39:29.812 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-idna. 2024-09-13T17:39:29.828 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../051-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-13T17:39:29.871 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T17:39:30.182 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-urllib3. 2024-09-13T17:39:30.208 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-13T17:39:30.248 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T17:39:30.585 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests. 2024-09-13T17:39:30.612 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-13T17:39:30.651 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T17:39:30.937 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-13T17:39:30.964 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-13T17:39:30.996 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T17:39:31.265 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-13T17:39:31.280 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:31.323 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:31.695 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-13T17:39:31.722 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:31.761 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:32.073 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-13T17:39:32.100 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../057-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-13T17:39:32.139 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-13T17:39:32.433 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-openssl. 2024-09-13T17:39:32.460 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../058-python3-openssl_21.0.0-1_all.deb ... 2024-09-13T17:39:32.500 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-13T17:39:32.778 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr. 2024-09-13T17:39:32.804 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../059-ceph-mgr_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:32.836 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:33.247 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mon. 2024-09-13T17:39:33.274 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../060-ceph-mon_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:33.330 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mon (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:33.951 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-osd. 2024-09-13T17:39:33.978 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../061-ceph-osd_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:34.017 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-osd (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:35.104 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph. 2024-09-13T17:39:35.119 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../062-ceph_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:35.154 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:35.456 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-fuse. 2024-09-13T17:39:35.483 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../063-ceph-fuse_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:35.523 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-fuse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:35.884 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mds. 2024-09-13T17:39:35.911 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../064-ceph-mds_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:35.951 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mds (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:36.296 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephadm. 2024-09-13T17:39:36.322 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../065-cephadm_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:36.354 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephadm (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:36.732 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-13T17:39:36.758 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../066-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-13T17:39:36.806 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-13T17:39:37.159 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-13T17:39:37.175 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../067-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T17:39:37.218 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T17:39:38.231 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-babel. 2024-09-13T17:39:38.259 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../068-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-13T17:39:38.297 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T17:39:39.334 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-jinja2. 2024-09-13T17:39:39.361 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../069-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-13T17:39:39.400 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T17:39:39.721 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-13T17:39:39.747 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../070-ceph-mgr-cephadm_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:40.116 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pluggy. 2024-09-13T17:39:40.142 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-13T17:39:40.173 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-13T17:39:40.460 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-py. 2024-09-13T17:39:40.486 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2024-09-13T17:39:40.526 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-13T17:39:40.872 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-13T17:39:40.898 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-13T17:39:40.938 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-13T17:39:41.242 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-toml. 2024-09-13T17:39:41.294 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2024-09-13T17:39:41.308 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-13T17:39:41.611 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pytest. 2024-09-13T17:39:41.637 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-13T17:39:41.676 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T17:39:42.022 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-13T17:39:42.049 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-13T17:39:42.088 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-13T17:39:42.391 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-routes. 2024-09-13T17:39:42.418 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-13T17:39:42.458 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T17:39:42.795 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-13T17:39:42.822 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:42.861 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:44.138 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-13T17:39:44.161 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-13T17:39:44.208 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T17:39:44.903 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-joblib. 2024-09-13T17:39:44.930 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-13T17:39:44.969 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T17:39:45.272 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-13T17:39:45.285 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-13T17:39:45.322 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-13T17:39:45.625 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-sklearn. 2024-09-13T17:39:45.652 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-13T17:39:45.691 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T17:39:46.452 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-13T17:39:46.480 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:46.518 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:47.438 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-cachetools. 2024-09-13T17:39:47.465 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-13T17:39:47.504 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-13T17:39:47.815 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-13T17:39:47.843 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-13T17:39:47.882 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-13T17:39:48.202 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-13T17:39:48.230 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-13T17:39:48.269 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T17:39:48.572 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-rsa. 2024-09-13T17:39:48.599 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-13T17:39:48.630 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-13T17:39:48.917 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-google-auth. 2024-09-13T17:39:48.945 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-13T17:39:48.983 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-13T17:39:49.312 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-13T17:39:49.339 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-13T17:39:49.378 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T17:39:49.689 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-websocket. 2024-09-13T17:39:49.716 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-13T17:39:49.756 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-13T17:39:50.068 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-13T17:39:50.095 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-13T17:39:50.133 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T17:39:50.789 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-13T17:39:50.818 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:50.855 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:51.108 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-13T17:39:51.136 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-13T17:39:51.175 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T17:39:51.461 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-13T17:39:51.489 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T17:39:51.528 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T17:39:51.789 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package jq. 2024-09-13T17:39:51.816 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-13T17:39:51.855 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-13T17:39:52.108 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package socat. 2024-09-13T17:39:52.124 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-13T17:39:52.166 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-13T17:39:52.511 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package xmlstarlet. 2024-09-13T17:39:52.527 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-13T17:39:52.577 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-13T17:39:52.932 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-test. 2024-09-13T17:39:52.959 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../098-ceph-test_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:52.998 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-test (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:55.136 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ceph-volume. 2024-09-13T17:39:55.164 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../099-ceph-volume_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:55.202 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ceph-volume (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:55.556 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-13T17:39:55.584 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:55.622 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-mirror (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:55.901 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-colorama. 2024-09-13T17:39:55.928 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2024-09-13T17:39:55.967 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-13T17:39:56.228 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-shell. 2024-09-13T17:39:56.244 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../102-cephfs-shell_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:56.286 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-shell (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:56.547 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package cephfs-top. 2024-09-13T17:39:56.563 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../103-cephfs-top_18.2.4-860-g6c6b083f-1jammy_all.deb ... 2024-09-13T17:39:56.605 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking cephfs-top (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:56.842 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-13T17:39:56.869 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:39:56.908 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libcephfs-dev (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:39:57.269 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package nvme-cli. 2024-09-13T17:39:57.297 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-13T17:39:57.336 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T17:39:57.657 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-13T17:39:57.685 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-13T17:39:57.723 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-13T17:39:58.137 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pastescript. 2024-09-13T17:39:58.163 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-13T17:39:58.201 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-13T17:39:58.481 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-psutil. 2024-09-13T17:39:58.498 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-13T17:39:58.531 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-13T17:39:58.860 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pygments. 2024-09-13T17:39:58.887 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-13T17:39:58.926 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T17:39:59.355 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-13T17:39:59.384 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-13T17:39:59.422 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-13T17:39:59.733 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package python3-simplejson. 2024-09-13T17:39:59.761 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-13T17:39:59.799 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-13T17:40:00.102 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-13T17:40:00.130 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-13T17:40:00.169 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-13T17:40:00.898 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package radosgw. 2024-09-13T17:40:00.926 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../113-radosgw_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:40:00.964 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking radosgw (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:01.701 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package rbd-fuse. 2024-09-13T17:40:01.728 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../114-rbd-fuse_18.2.4-860-g6c6b083f-1jammy_amd64.deb ... 2024-09-13T17:40:01.767 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking rbd-fuse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:02.055 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-13T17:40:02.306 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-13T17:40:02.406 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-13T17:40:02.646 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-13T17:40:02.955 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-13T17:40:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-13T17:40:03.634 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephadm (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:04.168 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user cephadm....done 2024-09-13T17:40:04.307 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-13T17:40:04.592 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-13T17:40:04.860 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-13T17:40:04.986 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-13T17:40:05.488 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-13T17:40:05.738 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-13T17:40:05.839 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-13T17:40:06.124 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-13T17:40:06.493 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-13T17:40:06.736 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-13T17:40:07.004 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-13T17:40:07.280 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-13T17:40:07.531 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-13T17:40:07.807 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-13T17:40:08.058 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-argparse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:08.309 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-13T17:40:08.435 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-13T17:40:08.687 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-13T17:40:08.938 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-13T17:40:09.039 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-13T17:40:09.382 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-13T17:40:09.602 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-13T17:40:09.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-13T17:40:10.060 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-13T17:40:10.370 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-13T17:40:11.025 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-13T17:40:11.265 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-13T17:40:11.391 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-13T17:40:11.542 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-13T17:40:11.834 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-13T17:40:12.211 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-13T17:40:12.471 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-13T17:40:12.814 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-13T17:40:13.099 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-13T17:40:13.401 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-13T17:40:13.735 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-13T17:40:13.986 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-13T17:40:14.112 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-ceph-common (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:14.422 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-13T17:40:14.739 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-13T17:40:15.920 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:40:16.055 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-13T17:40:16.314 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-13T17:40:16.449 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-13T17:40:16.591 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-13T17:40:16.876 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-13T17:40:17.119 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-13T17:40:17.362 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:40:17.488 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2024-09-13T17:40:17.614 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-13T17:40:17.924 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-13T17:40:18.175 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-13T17:40:18.435 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-13T17:40:18.669 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-13T17:40:18.912 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-13T17:40:19.180 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-13T17:40:19.281 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-13T17:40:19.557 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-13T17:40:19.683 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-13T17:40:19.976 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-13T17:40:20.268 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-13T17:40:20.520 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-13T17:40:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-13T17:40:21.156 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-13T17:40:21.282 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-13T17:40:21.651 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-13T17:40:21.893 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-13T17:40:22.153 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-13T17:40:22.279 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-13T17:40:22.580 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-13T17:40:22.873 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-13T17:40:23.158 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-13T17:40:23.284 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-13T17:40:23.543 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-13T17:40:23.828 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-13T17:40:24.079 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-13T17:40:24.456 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-13T17:40:24.740 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-13T17:40:25.076 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-13T17:40:25.202 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librados2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:25.336 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librgw2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:25.673 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libsqlite3-mod-ceph (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.029 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs2 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.150 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libradosstriper1 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.251 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-13T17:40:26.537 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up librbd1 (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-modules-core (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.747 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-fuse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:26.994 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:40:26.994 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-13T17:40:27.540 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libcephfs-dev (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:27.666 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rados (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:27.792 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-top (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:27.918 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-13T17:40:29.131 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rbd (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:29.257 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up rbd-fuse (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:29.383 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-rgw (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:29.484 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up python3-cephfs (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:29.635 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-shell (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-common (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:30.307 INFO:teuthology.orchestra.run.smithi096.stdout:Adding group ceph....done 2024-09-13T17:40:30.646 INFO:teuthology.orchestra.run.smithi096.stdout:Adding system user ceph....done 2024-09-13T17:40:30.809 INFO:teuthology.orchestra.run.smithi096.stdout:Setting system user ceph properties....done 2024-09-13T17:40:30.822 INFO:teuthology.orchestra.run.smithi096.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-13T17:40:30.959 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-13T17:40:31.305 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-13T17:40:31.813 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up cephfs-mirror (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:32.023 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:40:32.023 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-13T17:40:32.548 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-test (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:32.674 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up radosgw (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:33.139 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:40:33.139 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-13T17:40:33.576 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-base (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:33.895 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-13T17:40:34.354 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mds (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:34.580 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:34.580 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-13T17:40:35.064 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:35.280 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:35.280 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-13T17:40:35.756 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-osd (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:36.004 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:40:36.004 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-13T17:40:36.500 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-k8sevents (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:36.618 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-diskprediction-local (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:36.736 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mon (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:36.932 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:40:36.932 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-13T17:40:37.379 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-cephadm (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:37.530 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:37.657 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-mgr-dashboard (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:37.790 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ceph-volume (18.2.4-860-g6c6b083f-1jammy) ... 2024-09-13T17:40:38.118 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T17:40:39.181 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T17:40:39.649 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:40:39.652 DEBUG:teuthology.orchestra.run.smithi096:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-13T17:40:39.710 INFO:teuthology.orchestra.run.smithi096.stdout:Reading package lists... 2024-09-13T17:40:39.921 INFO:teuthology.orchestra.run.smithi096.stdout:Building dependency tree... 2024-09-13T17:40:39.922 INFO:teuthology.orchestra.run.smithi096.stdout:Reading state information... 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages were automatically installed and are no longer required: 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-13T17:40:40.122 INFO:teuthology.orchestra.run.smithi096.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-13T17:40:40.123 INFO:teuthology.orchestra.run.smithi096.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-13T17:40:40.123 INFO:teuthology.orchestra.run.smithi096.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-13T17:40:40.123 INFO:teuthology.orchestra.run.smithi096.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-13T17:40:40.124 INFO:teuthology.orchestra.run.smithi096.stdout:The following additional packages will be installed: 2024-09-13T17:40:40.124 INFO:teuthology.orchestra.run.smithi096.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-13T17:40:40.124 INFO:teuthology.orchestra.run.smithi096.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-13T17:40:40.125 INFO:teuthology.orchestra.run.smithi096.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-13T17:40:40.125 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-13T17:40:40.125 INFO:teuthology.orchestra.run.smithi096.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T17:40:40.126 INFO:teuthology.orchestra.run.smithi096.stdout:Suggested packages: 2024-09-13T17:40:40.126 INFO:teuthology.orchestra.run.smithi096.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-13T17:40:40.126 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-13T17:40:40.126 INFO:teuthology.orchestra.run.smithi096.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-13T17:40:40.166 INFO:teuthology.orchestra.run.smithi096.stdout:The following NEW packages will be installed: 2024-09-13T17:40:40.166 INFO:teuthology.orchestra.run.smithi096.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-13T17:40:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-13T17:40:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-13T17:40:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-13T17:40:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-13T17:40:40.167 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-13T17:40:40.168 INFO:teuthology.orchestra.run.smithi096.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-13T17:40:40.169 INFO:teuthology.orchestra.run.smithi096.stdout:The following packages will be upgraded: 2024-09-13T17:40:40.170 INFO:teuthology.orchestra.run.smithi096.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-13T17:40:40.222 INFO:teuthology.orchestra.run.smithi096.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-13T17:40:40.222 INFO:teuthology.orchestra.run.smithi096.stdout:Need to get 55.8 MB of archives. 2024-09-13T17:40:40.223 INFO:teuthology.orchestra.run.smithi096.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-13T17:40:40.223 INFO:teuthology.orchestra.run.smithi096.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-13T17:40:40.358 INFO:teuthology.orchestra.run.smithi096.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-13T17:40:40.366 INFO:teuthology.orchestra.run.smithi096.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-13T17:40:40.406 INFO:teuthology.orchestra.run.smithi096.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-13T17:40:40.406 INFO:teuthology.orchestra.run.smithi096.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-13T17:40:40.407 INFO:teuthology.orchestra.run.smithi096.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-13T17:40:40.418 INFO:teuthology.orchestra.run.smithi096.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-13T17:40:40.418 INFO:teuthology.orchestra.run.smithi096.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-13T17:40:40.420 INFO:teuthology.orchestra.run.smithi096.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-13T17:40:40.422 INFO:teuthology.orchestra.run.smithi096.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-13T17:40:40.422 INFO:teuthology.orchestra.run.smithi096.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-13T17:40:40.424 INFO:teuthology.orchestra.run.smithi096.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-13T17:40:40.425 INFO:teuthology.orchestra.run.smithi096.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-13T17:40:40.442 INFO:teuthology.orchestra.run.smithi096.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-13T17:40:40.442 INFO:teuthology.orchestra.run.smithi096.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-13T17:40:40.443 INFO:teuthology.orchestra.run.smithi096.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-13T17:40:40.703 INFO:teuthology.orchestra.run.smithi096.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-13T17:40:40.705 INFO:teuthology.orchestra.run.smithi096.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-13T17:40:40.706 INFO:teuthology.orchestra.run.smithi096.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-13T17:40:40.792 INFO:teuthology.orchestra.run.smithi096.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-13T17:40:40.792 INFO:teuthology.orchestra.run.smithi096.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-13T17:40:40.796 INFO:teuthology.orchestra.run.smithi096.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-13T17:40:40.798 INFO:teuthology.orchestra.run.smithi096.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-13T17:40:40.804 INFO:teuthology.orchestra.run.smithi096.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-13T17:40:40.804 INFO:teuthology.orchestra.run.smithi096.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-13T17:40:40.824 INFO:teuthology.orchestra.run.smithi096.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-13T17:40:40.824 INFO:teuthology.orchestra.run.smithi096.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-13T17:40:40.834 INFO:teuthology.orchestra.run.smithi096.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-13T17:40:40.852 INFO:teuthology.orchestra.run.smithi096.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-13T17:40:40.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-13T17:40:40.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-13T17:40:40.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-13T17:40:40.856 INFO:teuthology.orchestra.run.smithi096.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-13T17:40:40.867 INFO:teuthology.orchestra.run.smithi096.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-13T17:40:40.869 INFO:teuthology.orchestra.run.smithi096.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-13T17:40:41.086 INFO:teuthology.orchestra.run.smithi096.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-13T17:40:41.098 INFO:teuthology.orchestra.run.smithi096.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-13T17:40:41.098 INFO:teuthology.orchestra.run.smithi096.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-13T17:40:41.506 INFO:teuthology.orchestra.run.smithi096.stdout:Preconfiguring packages ... 2024-09-13T17:40:41.657 INFO:teuthology.orchestra.run.smithi096.stdout:Fetched 55.8 MB in 1s (60.5 MB/s) 2024-09-13T17:40:41.992 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173232 files and directories currently installed.) 2024-09-13T17:40:41.997 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T17:40:42.185 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T17:40:42.789 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-13T17:40:43.709 INFO:teuthology.orchestra.run.smithi096.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-13T17:40:43.941 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package flex. 2024-09-13T17:40:44.034 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173232 files and directories currently installed.) 2024-09-13T17:40:44.039 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-13T17:40:44.073 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-13T17:40:44.598 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T17:40:44.770 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-13T17:40:45.180 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T17:40:45.697 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173305 files and directories currently installed.) 2024-09-13T17:40:45.702 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T17:40:45.845 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T17:40:46.270 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-13T17:40:46.385 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-13T17:40:46.897 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T17:40:47.249 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package bison. 2024-09-13T17:40:47.348 INFO:teuthology.orchestra.run.smithi096.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173305 files and directories currently installed.) 2024-09-13T17:40:47.354 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-13T17:40:47.392 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T17:40:47.709 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-13T17:40:47.725 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-13T17:40:47.768 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T17:40:48.096 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package dump. 2024-09-13T17:40:48.123 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-13T17:40:48.163 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking dump (0.4b47-3) ... 2024-09-13T17:40:48.441 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package indent. 2024-09-13T17:40:48.457 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:40:48.491 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T17:40:48.702 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-13T17:40:48.715 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-13T17:40:48.744 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T17:40:49.073 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-13T17:40:49.101 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-13T17:40:49.140 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T17:40:49.393 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-13T17:40:49.420 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-13T17:40:49.459 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T17:40:49.705 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-13T17:40:49.733 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-13T17:40:49.763 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T17:40:50.032 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-13T17:40:50.060 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-13T17:40:50.099 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T17:40:50.343 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libjson-perl. 2024-09-13T17:40:50.371 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-13T17:40:50.410 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-13T17:40:50.697 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-13T17:40:50.724 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-13T17:40:50.763 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T17:40:52.542 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-13T17:40:52.569 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:40:52.608 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T17:40:52.870 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-13T17:40:52.897 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-13T17:40:52.936 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T17:40:53.314 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-13T17:40:53.341 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T17:40:53.381 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T17:40:53.717 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-13T17:40:53.744 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-13T17:40:53.784 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T17:40:54.104 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-13T17:40:54.131 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T17:40:54.170 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T17:40:54.515 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-13T17:40:54.543 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:40:54.582 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:40:54.910 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-13T17:40:54.937 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:40:54.976 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T17:40:55.263 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-13T17:40:55.279 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:40:55.321 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T17:40:55.573 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-13T17:40:55.601 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-13T17:40:55.641 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T17:40:56.119 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-13T17:40:56.146 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-13T17:40:56.177 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T17:40:56.522 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package modemmanager. 2024-09-13T17:40:56.550 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-13T17:40:56.589 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T17:40:57.714 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager. 2024-09-13T17:40:57.741 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-13T17:40:57.881 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-13T17:40:58.555 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package ppp. 2024-09-13T17:40:58.583 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-13T17:40:58.880 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T17:40:59.176 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package pptp-linux. 2024-09-13T17:40:59.204 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-13T17:40:59.235 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-13T17:40:59.488 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-13T17:40:59.516 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-13T17:40:59.554 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-13T17:40:59.858 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-13T17:40:59.885 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-13T17:40:59.924 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client-common (238) ... 2024-09-13T17:41:00.203 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-13T17:41:00.231 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:41:00.269 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:41:00.858 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-common. 2024-09-13T17:41:00.886 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-13T17:41:00.943 INFO:teuthology.orchestra.run.smithi096.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-13T17:41:00.964 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-common (238) ... 2024-09-13T17:41:01.449 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-14. 2024-09-13T17:41:01.477 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-13T17:41:01.519 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:41:02.905 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql. 2024-09-13T17:41:02.922 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-13T17:41:02.963 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql (14+238) ... 2024-09-13T17:41:03.207 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-client. 2024-09-13T17:41:03.236 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-13T17:41:03.274 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-client (14+238) ... 2024-09-13T17:41:03.534 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-13T17:41:03.551 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-13T17:41:03.593 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-13T17:41:03.879 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package tcl8.6. 2024-09-13T17:41:03.907 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-13T17:41:03.946 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T17:41:04.206 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package tcl. 2024-09-13T17:41:04.235 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-13T17:41:04.278 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-13T17:41:04.626 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package wpasupplicant. 2024-09-13T17:41:04.654 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-13T17:41:04.692 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T17:41:05.039 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-13T17:41:05.090 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-13T17:41:05.091 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-13T17:41:05.475 INFO:teuthology.orchestra.run.smithi096.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-13T17:41:05.503 INFO:teuthology.orchestra.run.smithi096.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-13T17:41:05.548 INFO:teuthology.orchestra.run.smithi096.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T17:41:05.847 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client-common (238) ... 2024-09-13T17:41:06.040 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-13T17:41:06.141 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-13T17:41:06.267 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:41:06.393 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-13T17:41:06.519 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-13T17:41:06.654 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:41:07.154 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-13T17:41:07.235 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-13T17:41:07.605 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:41:07.606 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-13T17:41:08.088 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-13T17:41:08.214 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-13T17:41:08.331 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-13T17:41:08.432 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-13T17:41:08.550 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-13T17:41:08.676 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-13T17:41:08.802 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-13T17:41:08.928 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-13T17:41:09.600 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-13T17:41:09.710 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up dump (0.4b47-3) ... 2024-09-13T17:41:09.787 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-13T17:41:09.853 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T17:41:09.979 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-13T17:41:10.105 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-13T17:41:10.231 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-13T17:41:10.357 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-13T17:41:10.447 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-13T17:41:10.517 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-13T17:41:10.626 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-13T17:41:10.788 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-13T17:41:10.921 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T17:41:11.047 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-13T17:41:11.215 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-client (14+238) ... 2024-09-13T17:41:11.341 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-13T17:41:11.798 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:11.798 INFO:teuthology.orchestra.run.smithi096.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-13T17:41:11.799 INFO:teuthology.orchestra.run.smithi096.stdout:which means they are currently configured by ifupdown: 2024-09-13T17:41:11.799 INFO:teuthology.orchestra.run.smithi096.stdout:- enp3s0f1 2024-09-13T17:41:11.799 INFO:teuthology.orchestra.run.smithi096.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-13T17:41:11.799 INFO:teuthology.orchestra.run.smithi096.stdout:remove their configuration from /etc/network/interfaces. 2024-09-13T17:41:11.799 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:11.927 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-13T17:41:12.234 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-13T17:41:12.545 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-13T17:41:13.019 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-13T17:41:13.529 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-13T17:41:13.955 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-13T17:41:14.073 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-common (238) ... 2024-09-13T17:41:14.900 INFO:teuthology.orchestra.run.smithi096.stdout:Adding user postgres to group ssl-cert 2024-09-13T17:41:15.126 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:15.126 INFO:teuthology.orchestra.run.smithi096.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-13T17:41:15.331 INFO:teuthology.orchestra.run.smithi096.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-13T17:41:15.331 INFO:teuthology.orchestra.run.smithi096.stdout:Removing obsolete dictionary files: 2024-09-13T17:41:15.865 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-13T17:41:16.233 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-13T17:41:16.559 INFO:teuthology.orchestra.run.smithi096.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-13T17:41:16.608 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:This user must also own the server process. 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:The database cluster will be initialized with locale "en_US". 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:The default text search configuration will be set to "english". 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:16.660 INFO:teuthology.orchestra.run.smithi096.stdout:Data page checksums are disabled. 2024-09-13T17:41:16.661 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-13T17:41:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:creating subdirectories ... ok 2024-09-13T17:41:16.661 INFO:teuthology.orchestra.run.smithi096.stdout:selecting dynamic shared memory implementation ... posix 2024-09-13T17:41:16.676 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default max_connections ... 100 2024-09-13T17:41:16.696 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default shared_buffers ... 128MB 2024-09-13T17:41:16.706 INFO:teuthology.orchestra.run.smithi096.stdout:selecting default time zone ... Etc/UTC 2024-09-13T17:41:16.707 INFO:teuthology.orchestra.run.smithi096.stdout:creating configuration files ... ok 2024-09-13T17:41:16.824 INFO:teuthology.orchestra.run.smithi096.stdout:running bootstrap script ... ok 2024-09-13T17:41:17.440 INFO:teuthology.orchestra.run.smithi096.stdout:performing post-bootstrap initialization ... ok 2024-09-13T17:41:18.078 INFO:teuthology.orchestra.run.smithi096.stdout:syncing data to disk ... ok 2024-09-13T17:41:18.455 INFO:teuthology.orchestra.run.smithi096.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-13T17:41:21.165 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T17:41:21.282 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-13T17:41:21.400 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up postgresql (14+238) ... 2024-09-13T17:41:21.526 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-13T17:41:21.652 INFO:teuthology.orchestra.run.smithi096.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-13T17:41:21.915 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:41:21.916 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-13T17:41:22.497 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-13T17:41:22.690 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-13T17:41:22.984 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-13T17:41:23.136 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-13T17:41:24.439 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-13T17:41:24.539 INFO:teuthology.orchestra.run.smithi096.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-13T17:41:25.176 INFO:teuthology.orchestra.run.smithi096.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-13T17:41:25.182 DEBUG:teuthology.parallel:result is None 2024-09-13T17:41:25.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3 2024-09-13T17:41:25.336 DEBUG:teuthology.orchestra.run.smithi096:> dpkg-query -W -f '${Version}' ceph 2024-09-13T17:41:25.351 INFO:teuthology.orchestra.run.smithi096.stdout:18.2.4-860-g6c6b083f-1jammy 2024-09-13T17:41:25.351 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-860-g6c6b083f-1jammy 2024-09-13T17:41:25.351 INFO:teuthology.task.install:The correct ceph version 18.2.4-860-g6c6b083f-1jammy is installed. 2024-09-13T17:41:25.353 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T17:41:25.353 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:25.353 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T17:41:25.404 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T17:41:25.405 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:25.405 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T17:41:25.459 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T17:41:25.511 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T17:41:25.512 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:25.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T17:41:25.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T17:41:25.616 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T17:41:25.617 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:25.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T17:41:25.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T17:41:25.728 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T17:41:25.810 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T17:41:25.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /var/log/ceph 2024-09-13T17:41:25.823 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T17:41:25.823 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T17:41:25.876 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T17:41:25.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T17:41:25.937 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-13T17:41:25.938 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-13T17:41:25.938 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-13T17:41:25.938 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:25.938 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-13T17:41:25.981 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-13T17:41:26.059 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T17:41:26.059 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T17:41:26.060 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909', 'branch': 'wip-rishabh-testing-20240910.100730-reef', 'description': 'fs/top/{begin/{0-install 1-ceph 2-logrotate 3-modules} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/fstop}', 'email': None, 'first_in_suite': False, 'job_id': '7903909', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'meta': [{'desc': '1 ceph cluster with 1 mon, 1 mgr, 3 osds, 2 mds, 2 clients'}], 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi', 'no_nested_subset': False, 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240910.100730-reef'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240910.100730-reef', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'mds.a', 'mds.b', 'osd.0', 'osd.1', 'osd.2', 'client.0', 'client.1']], 'seed': 6439, 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'sleep_before_teardown': 0, 'subset': '964/1024', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240910.100730-reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi096.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH222ygDrwd7MpLaLNWW3/ck6qjAjg4Q2jbxdAMpTI4+At5hI0iIlfvaaaBDeDEszA0pRkfzuYP53TxfOnL9XNM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_fstop']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-13_14:42:16', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T17:41:26.060 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T17:41:26.105 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T17:41:26.161 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:26.161 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T17:41:26.209 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T17:41:26.209 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-13T17:41:26.257 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T17:41:26.257 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:26.257 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 891 Links: 1 2024-09-13T17:41:26.257 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:26.257 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-13 17:35:26.745160607 +0000 2024-09-13T17:41:26.258 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-13 17:35:26.509162715 +0000 2024-09-13T17:41:26.258 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-13 17:35:26.509162715 +0000 2024-09-13T17:41:26.258 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-13T17:41:26.258 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T17:41:26.311 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-13T17:41:26.311 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-13T17:41:26.311 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00045436 s, 1.1 MB/s 2024-09-13T17:41:26.312 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T17:41:26.362 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-13T17:41:26.408 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T17:41:26.408 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-13 17:35:26.965158642 +0000 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-13 17:35:26.961158677 +0000 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-13 17:35:26.961158677 +0000 2024-09-13T17:41:26.409 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-13T17:41:26.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T17:41:26.463 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-13T17:41:26.463 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-13T17:41:26.463 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000418427 s, 1.2 MB/s 2024-09-13T17:41:26.464 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T17:41:26.514 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-13T17:41:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T17:41:26.565 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:26.565 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-09-13T17:41:26.566 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:26.566 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-13 17:35:27.741151710 +0000 2024-09-13T17:41:26.566 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-13 17:35:27.469154139 +0000 2024-09-13T17:41:26.566 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-13 17:35:27.469154139 +0000 2024-09-13T17:41:26.566 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-13T17:41:26.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T17:41:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-13T17:41:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-13T17:41:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000395229 s, 1.3 MB/s 2024-09-13T17:41:26.620 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T17:41:26.669 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-13T17:41:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T17:41:26.717 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 944 Links: 1 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-13 17:35:28.221147423 +0000 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-13 17:35:27.989149495 +0000 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-13 17:35:27.989149495 +0000 2024-09-13T17:41:26.718 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: - 2024-09-13T17:41:26.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T17:41:26.770 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-13T17:41:26.770 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-13T17:41:26.770 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000372629 s, 1.4 MB/s 2024-09-13T17:41:26.772 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T17:41:26.821 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T17:41:26.822 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-13T17:41:26.822 INFO:tasks.ceph:Generating config... 2024-09-13T17:41:26.823 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-13T17:41:26.823 INFO:tasks.ceph:[client] debug client = 20 2024-09-13T17:41:26.823 INFO:tasks.ceph:[client] debug ms = 1 2024-09-13T17:41:26.823 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-13T17:41:26.823 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-13T17:41:26.823 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-13T17:41:26.823 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-13T17:41:26.823 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-13T17:41:26.823 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-13T17:41:26.824 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T17:41:26.825 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T17:41:26.826 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T17:41:26.826 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T17:41:26.826 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-13T17:41:26.826 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T17:41:26.826 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:26.896 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T17:41:26.912 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T17:41:26.996 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:41:27.051 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.96')] 2024-09-13T17:41:27.052 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.96'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-13T17:41:27.053 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:27.053 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:41:27.104 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.96 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:27.182 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:27.182 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: generated fsid a54aa7b1-da4c-43a3-b94f-1607e77cbcc4 2024-09-13T17:41:27.182 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = pacific 2024-09-13T17:41:27.182 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 0 2024-09-13T17:41:27.182 INFO:teuthology.orchestra.run.smithi096.stdout:fsid a54aa7b1-da4c-43a3-b94f-1607e77cbcc4 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:last_changed 2024-09-13T17:41:27.184515+0000 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-09-13T17:41:27.184515+0000 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:min_mon_release 16 (pacific) 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:election_strategy: 1 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-09-13T17:41:27.183 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-13T17:41:27.184 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T17:41:27.232 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID a54aa7b1-da4c-43a3-b94f-1607e77cbcc4... 2024-09-13T17:41:27.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T17:41:27.303 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-09-13T17:41:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: chdir = "" 2024-09-13T17:41:27.303 INFO:teuthology.orchestra.run.smithi096.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: auth supported = cephx 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: filestore xattr use omap = true 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: mon clock drift allowed = 1.000 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: osd crush chooseleaf type = 0 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: auth debug = true 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on old message = true 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: ms die on bug = true 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.304 INFO:teuthology.orchestra.run.smithi096.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon pg warn max object skew = 0 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default size = 2 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow primary affinity = true 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd allow pg remap = true 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on legacy crush tunables = false 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on crush straw calc version zero = false 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on no sortbitwise = false 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on osd down out interval zero = false 2024-09-13T17:41:27.305 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on too few osds = false 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon_allow_pool_size_one = true 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: osd default data pool replay window = 5 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon allow pool delete = true 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon cluster log file level = debug 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: debug asserts on shutdown = true 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon health detail to clog = false 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: mon host = 172.21.15.96 2024-09-13T17:41:27.306 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = a54aa7b1-da4c-43a3-b94f-1607e77cbcc4 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout:[osd] 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd journal size = 100 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub load threshold = 5.0 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd scrub max interval = 600 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd mclock profile = high_recovery_ops 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd recover clone overlap = true 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd recovery max chunk = 1048576 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug shutdown = true 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug op order = true 2024-09-13T17:41:27.307 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug verify stray on activate = true 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: osd open classes on start = true 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug pg log writeout = true 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: osd deep scrub update digest min age = 30 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: osd map max advance = 10 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: journal zero on create = true 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: filestore ondisk finisher threads = 3 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: filestore apply finisher threads = 3 2024-09-13T17:41:27.308 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: bdev debug aio = true 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: osd debug misdirected ops = true 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: bdev async discard = True 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: bdev enable discard = True 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore allocator = bitmap 2024-09-13T17:41:27.309 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore block size = 96636764160 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: bluestore fsck on mount = True 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluefs = 1/20 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: debug bluestore = 1/20 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: debug osd = 20 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: debug rocksdb = 4/10 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd full ratio = 0.9 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd nearfull ratio = 0.8 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: osd failsafe full ratio = 0.95 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: osd objectstore = bluestore 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.310 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr] 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug mgr = 20 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: mgr/telemetry/nag = false 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout:[mon] 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug mon = 20 2024-09-13T17:41:27.311 INFO:teuthology.orchestra.run.smithi096.stdout: debug paxos = 20 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: debug auth = 20 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon data avail warn = 5 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon mgr mkfs grace = 240 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min pgs per osd = 4 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd reporter subtree level = osd 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon osd prime pg temp = true 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: mon reweight min bytes per osd = 10 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T17:41:27.312 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: # 1m isn't quite enough 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon_down_mkfs_grace = 2m 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon_warn_on_filestore_osds = false 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon op complaint time = 120 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: mon warn on pool no app = False 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout:[client] 2024-09-13T17:41:27.313 INFO:teuthology.orchestra.run.smithi096.stdout: rgw cache enabled = true 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable ops log = true 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: rgw enable usage log = true 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: client mount timeout = 600 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: debug client = 20 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 900 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 900 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout:[mon.a] 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2024-09-13T17:41:27.314 INFO:teuthology.orchestra.run.smithi096.stdout:[mds] 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds = 20 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: debug mds balancer = 20 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: debug ms = 1 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug frag = True 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: mds debug scatterstat = True 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: mds op complaint time = 180 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: mds verify scatter = True 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: osd op complaint time = 180 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: rados mon op timeout = 900 2024-09-13T17:41:27.315 INFO:teuthology.orchestra.run.smithi096.stdout: rados osd op timeout = 900 2024-09-13T17:41:27.316 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T17:41:27.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T17:41:27.404 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T17:41:27.404 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:27.404 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T17:41:27.453 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:27.453 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T17:41:27.501 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:41:27.501 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:27.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T17:41:27.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T17:41:27.568 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:27.568 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:27.617 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T17:41:27.617 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T17:41:27.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:41:27.702 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T17:41:27.705 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T17:41:27.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:41:27.789 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-13T17:41:27.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-13T17:41:27.844 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:41:27.928 INFO:teuthology.orchestra.run.smithi096.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-13T17:41:27.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-13T17:41:27.984 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T17:41:27.984 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T17:41:28.061 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T17:41:28.072 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T17:41:28.149 INFO:teuthology.orchestra.run.smithi096.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T17:41:28.160 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T17:41:28.160 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi096.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-13T17:41:28.160 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:28.216 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T17:41:28.216 INFO:tasks.ceph:role: osd.0 2024-09-13T17:41:28.216 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:41:28.217 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:28.280 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:28.281 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:41:28.281 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:28.281 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:28.772 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-13T17:41:28.774 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-13T17:41:28.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:28.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:28.936 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:41:28.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:28.938 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:28.991 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T17:41:28.991 INFO:tasks.ceph:role: osd.1 2024-09-13T17:41:28.992 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:41:28.992 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:29.056 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:29.057 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:41:29.057 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:29.057 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:29.495 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-13T17:41:29.496 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-13T17:41:29.496 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:29.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:29.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:29.597 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:41:29.598 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:29.653 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-13T17:41:29.653 INFO:tasks.ceph:role: osd.2 2024-09-13T17:41:29.654 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:41:29.654 DEBUG:teuthology.orchestra.run.smithi096:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T17:41:29.716 INFO:teuthology.orchestra.run.smithi096.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T17:41:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T17:41:29.716 INFO:teuthology.orchestra.run.smithi096.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: = sunit=0 swidth=0 blks 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T17:41:29.717 INFO:teuthology.orchestra.run.smithi096.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T17:41:30.168 INFO:teuthology.orchestra.run.smithi096.stdout:Discarding blocks...Done. 2024-09-13T17:41:30.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi096.front.sepia.ceph.com -o noatime 2024-09-13T17:41:30.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:30.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:30.266 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: /sbin/restorecon: command not found 2024-09-13T17:41:30.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:41:30.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.370+0000 7f1a106267c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T17:41:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.374+0000 7f1a106267c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T17:41:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.374+0000 7f1a106267c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.374+0000 7f1a106267c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:30.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.374+0000 7f1a106267c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T17:41:30.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:30.386+0000 7f1a106267c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T17:41:33.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T17:41:33.374 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:33.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T17:41:33.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T17:41:33.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:33.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:33.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T17:41:33.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:33.454+0000 7fbe47ef87c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T17:41:36.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T17:41:36.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:36.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T17:41:36.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T17:41:36.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:36.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:36.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T17:41:36.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:36.582+0000 7f451c0277c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T17:41:39.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T17:41:39.593 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T17:41:39.594 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T17:41:39.648 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.648 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-13T17:41:39.704 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-13T17:41:39.760 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T17:41:39.816 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.816 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T17:41:39.872 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T17:41:39.928 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.928 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T17:41:39.976 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:39.976 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T17:41:40.025 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T17:41:40.025 DEBUG:teuthology.orchestra.run.smithi096:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout:[mgr.x] 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBHeeRmOOcEKhAAJmefIyHpVdx4zDzhwQcawA== 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a] 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBHeeRmAVcqLxAAIRo/psndnU3y/VG+/goNRQ== 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b] 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBHeeRmpF6ANxAAzrn1jVVSCWKv5q1kdChbWA== 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.0] 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBKeeRmkXcoFhAAdX1HBKKYCHqtacYyAZbF2w== 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.1] 2024-09-13T17:41:40.079 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBNeeRmh189GxAAFDpo1k0lxmmillsXoiKXMA== 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout:[osd.2] 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBQeeRmKpvLIhAA4Y7okMAZAQ9cjYpkaKF/kA== 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBIeeRm3enLAxAAZGIwl+Bs06GY4JG0dbxLBQ== 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout:[client.1] 2024-09-13T17:41:40.080 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQBIeeRmNlIQCRAAFFcBZ0Pwtq8kXtBAAFkuDA== 2024-09-13T17:41:40.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T17:41:40.169 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:41:40.214 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-13T17:41:40.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:40.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:40.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T17:41:40.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:41:40.627 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T17:41:40.670 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T17:41:40.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T17:41:40.723 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T17:41:40.812 DEBUG:teuthology.orchestra.run.smithi096:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T17:41:40.867 DEBUG:teuthology.orchestra.run.smithi096:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T17:41:40.912 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T17:41:40.912 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T17:41:40.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T17:41:40.958 INFO:tasks.ceph.mon.a:Started 2024-09-13T17:41:40.958 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T17:41:40.958 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T17:41:40.958 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T17:41:40.963 INFO:tasks.ceph.mgr.x:Started 2024-09-13T17:41:40.963 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T17:41:40.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T17:41:41.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f60754ab640 1 Processor -- start 2024-09-13T17:41:41.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f60754ab640 1 -- start start 2024-09-13T17:41:41.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f60754ab640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f60754ab640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6070138050 con 0x7f607009d690 2024-09-13T17:41:41.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f60754ab640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607009c410 con 0x7f607013af50 2024-09-13T17:41:41.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34434/0 (socket says 172.21.15.96:34434) 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 -- 172.21.15.96:0/3201398193 learned_addr learned my addr 172.21.15.96:0/3201398193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606effd640 1 --1- 172.21.15.96:0/3201398193 >> v1:172.21.15.96:6789/0 conn(0x7f607009d690 0x7f6070138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39422/0 (socket says 172.21.15.96:39422) 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 -- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 msgr2=0x7f607013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 -- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 msgr2=0x7f607013d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 --2- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-13T17:41:41.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.082+0000 7f606e7fc640 1 --2- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:41:41.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281388654 0 0) 0x7f6060001120 con 0x7f607009d690 2024-09-13T17:41:41.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6058003530 con 0x7f607009d690 2024-09-13T17:41:41.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4129033268 0 0) 0x7f6058003530 con 0x7f607009d690 2024-09-13T17:41:41.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 msgr2=0x7f607013d340 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-13T17:41:41.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 --2- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 --> v1:172.21.15.96:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607009cba0 con 0x7f607009d690 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v1:172.21.15.96:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 597839753 0 0) 0x7f6060001740 con 0x7f607009d690 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 >> v1:172.21.15.96:6789/0 conn(0x7f607009d690 legacy=0x7f6070138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 --2- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f6058003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607009c410 con 0x7f60580039d0 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606effd640 1 --2- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f6058003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.102+0000 7f606effd640 1 -- 172.21.15.96:0/3201398193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6058003530 con 0x7f60580039d0 2024-09-13T17:41:41.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f606effd640 1 --2- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f6058003dd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6060007e40 tx=0x7f606001f270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806b6868157bb555 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60600a8e30 con 0x7f60580039d0 2024-09-13T17:41:41.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060007070 con 0x7f60580039d0 2024-09-13T17:41:41.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f606dffb640 1 -- 172.21.15.96:0/3201398193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6060007350 con 0x7f60580039d0 2024-09-13T17:41:41.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 msgr2=0x7f6058003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 --2- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f6058003dd0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6060007e40 tx=0x7f606001f270 comp rx=0 tx=0).stop 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- 172.21.15.96:0/3201398193 shutdown_connections 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 --2- 172.21.15.96:0/3201398193 >> v2:172.21.15.96:3300/0 conn(0x7f607013af50 0x7f607013d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 --2- 172.21.15.96:0/3201398193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f6058003dd0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- 172.21.15.96:0/3201398193 >> 172.21.15.96:0/3201398193 conn(0x7f607006bc60 msgr2=0x7f607006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- 172.21.15.96:0/3201398193 shutdown_connections 2024-09-13T17:41:41.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- 172.21.15.96:0/3201398193 wait complete. 2024-09-13T17:41:41.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 Processor -- start 2024-09-13T17:41:41.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.118+0000 7f60754ab640 1 -- start start 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60754ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60754ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700a4d10 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f606effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f606effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34466/0 (socket says 172.21.15.96:34466) 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f606effd640 1 -- 172.21.15.96:0/1636988850 learned_addr learned my addr 172.21.15.96:0/1636988850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f606effd640 1 -- 172.21.15.96:0/1636988850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6070137170 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f606effd640 1 --2- 172.21.15.96:0/1636988850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6060000c10 tx=0x7f6058005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6060007310 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6060007470 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60700a2c10 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60600047f0 con 0x7f60580039d0 2024-09-13T17:41:41.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60700a5020 con 0x7f60580039d0 2024-09-13T17:41:41.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f60600ae070 con 0x7f60580039d0 2024-09-13T17:41:41.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f60600b1930 con 0x7f60580039d0 2024-09-13T17:41:41.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.122+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60700a4e60 con 0x7f60580039d0 2024-09-13T17:41:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.126+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f6070138050 con 0x7f60580039d0 2024-09-13T17:41:41.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.190+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6070065eb0 con 0x7f60580039d0 2024-09-13T17:41:41.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60577fe640 1 -- 172.21.15.96:0/1636988850 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f60600b1b90 con 0x7f60580039d0 2024-09-13T17:41:41.193 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 msgr2=0x7f60701c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 --2- 172.21.15.96:0/1636988850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6060000c10 tx=0x7f6058005800 comp rx=0 tx=0).stop 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 shutdown_connections 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 --2- 172.21.15.96:0/1636988850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60580039d0 0x7f60701c94c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 >> 172.21.15.96:0/1636988850 conn(0x7f607006bc60 msgr2=0x7f6070131320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 shutdown_connections 2024-09-13T17:41:41.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.194+0000 7f60754ab640 1 -- 172.21.15.96:0/1636988850 wait complete. 2024-09-13T17:41:41.211 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T17:41:41.211 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T17:41:41.211 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:41.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T17:41:41.223 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:41.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T17:41:41.274 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:41:41.274 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T17:41:41.327 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 554ada03-d5e2-4466-95ee-96adb96f409f 0 2024-09-13T17:41:41.342 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:41.342+0000 7f14b9f7d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:41:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dd1803640 1 Processor -- start 2024-09-13T17:41:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dd1803640 1 -- start start 2024-09-13T17:41:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dd1803640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1dcc09c180 0x7f1dcc13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dd1803640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc1326f0 con 0x7f1dcc132940 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dd1803640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc132500 con 0x7f1dcc09c180 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dca7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1dcc09c180 0x7f1dcc13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dcaffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1dcc132940 0x7f1dcc132d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39436/0 (socket says 172.21.15.96:39436) 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dcaffd640 1 -- 172.21.15.96:0/167761900 learned_addr learned my addr 172.21.15.96:0/167761900 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dca7fc640 1 -- 172.21.15.96:0/167761900 >> v1:172.21.15.96:6789/0 conn(0x7f1dcc132940 legacy=0x7f1dcc132d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dc9ffb640 1 -- 172.21.15.96:0/167761900 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152458913 0 0) 0x7f1dcc1326f0 con 0x7f1dcc132940 2024-09-13T17:41:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.458+0000 7f1dca7fc640 1 -- 172.21.15.96:0/167761900 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc09cfd0 con 0x7f1dcc09c180 2024-09-13T17:41:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dca7fc640 1 --2- 172.21.15.96:0/167761900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc13ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009740 tx=0x7f1dc001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c593052d583d651 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dc9ffb640 1 -- 172.21.15.96:0/167761900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc00ab400 con 0x7f1dcc09c180 2024-09-13T17:41:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dc9ffb640 1 -- 172.21.15.96:0/167761900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc00a7050 con 0x7f1dcc09c180 2024-09-13T17:41:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dc9ffb640 1 -- 172.21.15.96:0/167761900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dc00ab6b0 con 0x7f1dcc09c180 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- 172.21.15.96:0/167761900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 msgr2=0x7f1dcc13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 --2- 172.21.15.96:0/167761900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc13ae50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009740 tx=0x7f1dc001c390 comp rx=0 tx=0).stop 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- 172.21.15.96:0/167761900 shutdown_connections 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 --2- 172.21.15.96:0/167761900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc13ae50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- 172.21.15.96:0/167761900 >> 172.21.15.96:0/167761900 conn(0x7f1dcc06e020 msgr2=0x7f1dcc070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- 172.21.15.96:0/167761900 shutdown_connections 2024-09-13T17:41:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- 172.21.15.96:0/167761900 wait complete. 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 Processor -- start 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- start start 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dd1803640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc134650 con 0x7f1dcc09c180 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dcaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dcaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34480/0 (socket says 172.21.15.96:34480) 2024-09-13T17:41:41.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dcaffd640 1 -- 172.21.15.96:0/2078590264 learned_addr learned my addr 172.21.15.96:0/2078590264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dcaffd640 1 -- 172.21.15.96:0/2078590264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009420 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1dcaffd640 1 --2- 172.21.15.96:0/2078590264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00c420 tx=0x7f1dbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc0074d0 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dbc007630 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.462+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc004960 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc09f0e0 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc0a56d0 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f1dbc01d440 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1dbc01c6f0 con 0x7f1dcc09c180 2024-09-13T17:41:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc0a5250 con 0x7f1dcc09c180 2024-09-13T17:41:41.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.466+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f1dbc018070 con 0x7f1dcc09c180 2024-09-13T17:41:41.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.534+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "554ada03-d5e2-4466-95ee-96adb96f409f", "id": 0} v 0) v1 -- 0x7f1dcc134ba0 con 0x7f1dcc09c180 2024-09-13T17:41:41.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1db3fff640 1 -- 172.21.15.96:0/2078590264 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "554ada03-d5e2-4466-95ee-96adb96f409f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1dbc01ca20 con 0x7f1dcc09c180 2024-09-13T17:41:41.537 INFO:teuthology.orchestra.run.smithi096.stdout:0 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 msgr2=0x7f1dcc0a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 --2- 172.21.15.96:0/2078590264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00c420 tx=0x7f1dbc00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 shutdown_connections 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 --2- 172.21.15.96:0/2078590264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1dcc09c180 0x7f1dcc0a5e50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 >> 172.21.15.96:0/2078590264 conn(0x7f1dcc06e020 msgr2=0x7f1dcc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 shutdown_connections 2024-09-13T17:41:41.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.538+0000 7f1dd1803640 1 -- 172.21.15.96:0/2078590264 wait complete. 2024-09-13T17:41:41.554 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 97e27884-eab0-4aaa-b570-7baf268781e0 1 2024-09-13T17:41:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cf4df4640 1 Processor -- start 2024-09-13T17:41:41.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cf4df4640 1 -- start start 2024-09-13T17:41:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cf4df4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0cf0099150 0x7f0cf0099530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cf4df4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf0098a60 con 0x7f0cf009a230 2024-09-13T17:41:41.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cf4df4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf009ac20 con 0x7f0cf0099150 2024-09-13T17:41:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cee59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0cf0099150 0x7f0cf0099530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cedd9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0cf009a230 0x7f0cf0135da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39450/0 (socket says 172.21.15.96:39450) 2024-09-13T17:41:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cedd9b640 1 -- 172.21.15.96:0/722207869 learned_addr learned my addr 172.21.15.96:0/722207869 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cee59c640 1 -- 172.21.15.96:0/722207869 >> v1:172.21.15.96:6789/0 conn(0x7f0cf009a230 legacy=0x7f0cf0135da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.662+0000 7f0cee59c640 1 -- 172.21.15.96:0/722207869 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0098c90 con 0x7f0cf0099150 2024-09-13T17:41:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cee59c640 1 --2- 172.21.15.96:0/722207869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf0099530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009740 tx=0x7f0ce001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640e5dccc628717d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0ced59a640 1 -- 172.21.15.96:0/722207869 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0cf0136cc0 con 0x7f0cf0099150 2024-09-13T17:41:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0ced59a640 1 -- 172.21.15.96:0/722207869 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00a7050 con 0x7f0cf0099150 2024-09-13T17:41:41.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0ced59a640 1 -- 172.21.15.96:0/722207869 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ce00042f0 con 0x7f0cf0099150 2024-09-13T17:41:41.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 -- 172.21.15.96:0/722207869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 msgr2=0x7f0cf0099530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 --2- 172.21.15.96:0/722207869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf0099530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009740 tx=0x7f0ce001c390 comp rx=0 tx=0).stop 2024-09-13T17:41:41.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 -- 172.21.15.96:0/722207869 shutdown_connections 2024-09-13T17:41:41.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 --2- 172.21.15.96:0/722207869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf0099530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 -- 172.21.15.96:0/722207869 >> 172.21.15.96:0/722207869 conn(0x7f0cf009d020 msgr2=0x7f0cf009d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 -- 172.21.15.96:0/722207869 shutdown_connections 2024-09-13T17:41:41.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 -- 172.21.15.96:0/722207869 wait complete. 2024-09-13T17:41:41.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.666+0000 7f0cf4df4640 1 Processor -- start 2024-09-13T17:41:41.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 -- start start 2024-09-13T17:41:41.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00a2bd0 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cee59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cee59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34496/0 (socket says 172.21.15.96:34496) 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cee59c640 1 -- 172.21.15.96:0/2129187363 learned_addr learned my addr 172.21.15.96:0/2129187363 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cee59c640 1 -- 172.21.15.96:0/2129187363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0009420 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cee59c640 1 --2- 172.21.15.96:0/2129187363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0018fa0 tx=0x7f0ce0019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ce0003c90 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0003df0 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ce00ab570 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf00a1f40 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf00a03e0 con 0x7f0cf0099150 2024-09-13T17:41:41.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f0ce00b2040 con 0x7f0cf0099150 2024-09-13T17:41:41.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0ce00bebb0 con 0x7f0cf0099150 2024-09-13T17:41:41.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.670+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf0099530 con 0x7f0cf0099150 2024-09-13T17:41:41.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.674+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f0ce00dde30 con 0x7f0cf0099150 2024-09-13T17:41:41.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.742+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "97e27884-eab0-4aaa-b570-7baf268781e0", "id": 1} v 0) v1 -- 0x7f0cf013bef0 con 0x7f0cf0099150 2024-09-13T17:41:41.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.742+0000 7f0cdeffd640 1 -- 172.21.15.96:0/2129187363 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "97e27884-eab0-4aaa-b570-7baf268781e0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ce00cc070 con 0x7f0cf0099150 2024-09-13T17:41:41.743 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:41:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 msgr2=0x7f0cf00a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 --2- 172.21.15.96:0/2129187363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0018fa0 tx=0x7f0ce0019130 comp rx=0 tx=0).stop 2024-09-13T17:41:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 shutdown_connections 2024-09-13T17:41:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 --2- 172.21.15.96:0/2129187363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cf0099150 0x7f0cf00a2690 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 >> 172.21.15.96:0/2129187363 conn(0x7f0cf009d020 msgr2=0x7f0cf00708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 shutdown_connections 2024-09-13T17:41:41.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.746+0000 7f0cf4df4640 1 -- 172.21.15.96:0/2129187363 wait complete. 2024-09-13T17:41:41.759 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph osd new 5195b731-8e00-4da1-a5ad-ff98b219f282 2 2024-09-13T17:41:41.796 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:41.798+0000 7f14b9f7d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:41:41.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c6dada640 1 Processor -- start 2024-09-13T17:41:41.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c6dada640 1 -- start start 2024-09-13T17:41:41.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c6dada640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c6809dc40 0x7f4c68136270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c6dada640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6809ba60 con 0x7f4c681367b0 2024-09-13T17:41:41.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c6dada640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6809cf60 con 0x7f4c6809dc40 2024-09-13T17:41:41.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c677fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c6809dc40 0x7f4c68136270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c66ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4c681367b0 0x7f4c68099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39466/0 (socket says 172.21.15.96:39466) 2024-09-13T17:41:41.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c66ffd640 1 -- 172.21.15.96:0/1413247455 learned_addr learned my addr 172.21.15.96:0/1413247455 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:41.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c677fe640 1 -- 172.21.15.96:0/1413247455 >> v1:172.21.15.96:6789/0 conn(0x7f4c681367b0 legacy=0x7f4c68099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c677fe640 1 -- 172.21.15.96:0/1413247455 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6809c4e0 con 0x7f4c6809dc40 2024-09-13T17:41:41.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.870+0000 7f4c667fc640 1 -- 172.21.15.96:0/1413247455 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272988422 0 0) 0x7f4c5c001120 con 0x7f4c681367b0 2024-09-13T17:41:41.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 --2- 172.21.15.96:0/1413247455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c68136270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009740 tx=0x7f4c5801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87335f6a5ee7d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c667fc640 1 -- 172.21.15.96:0/1413247455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c580ab530 con 0x7f4c6809dc40 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c667fc640 1 -- 172.21.15.96:0/1413247455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580a7050 con 0x7f4c6809dc40 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c667fc640 1 -- 172.21.15.96:0/1413247455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c580ab7e0 con 0x7f4c6809dc40 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/1413247455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 msgr2=0x7f4c68136270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 --2- 172.21.15.96:0/1413247455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c68136270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009740 tx=0x7f4c5801c390 comp rx=0 tx=0).stop 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/1413247455 shutdown_connections 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 --2- 172.21.15.96:0/1413247455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c68136270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/1413247455 >> 172.21.15.96:0/1413247455 conn(0x7f4c6806f350 msgr2=0x7f4c68071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/1413247455 shutdown_connections 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/1413247455 wait complete. 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 Processor -- start 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- start start 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681ca080 con 0x7f4c6809dc40 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34500/0 (socket says 172.21.15.96:34500) 2024-09-13T17:41:41.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 -- 172.21.15.96:0/92242561 learned_addr learned my addr 172.21.15.96:0/92242561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 -- 172.21.15.96:0/92242561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58009420 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c677fe640 1 --2- 172.21.15.96:0/92242561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c58019fc0 tx=0x7f4c5801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c58004150 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c580042b0 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c68137190 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c58003620 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680a30b0 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f4c580b2050 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.874+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f4c580b5c90 con 0x7f4c6809dc40 2024-09-13T17:41:41.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.878+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c680a4090 con 0x7f4c6809dc40 2024-09-13T17:41:41.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.878+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f4c580dde30 con 0x7f4c6809dc40 2024-09-13T17:41:41.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.946+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5195b731-8e00-4da1-a5ad-ff98b219f282", "id": 2} v 0) v1 -- 0x7f4c68136270 con 0x7f4c6809dc40 2024-09-13T17:41:41.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c4ffff640 1 -- 172.21.15.96:0/92242561 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5195b731-8e00-4da1-a5ad-ff98b219f282", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c580cc030 con 0x7f4c6809dc40 2024-09-13T17:41:41.949 INFO:teuthology.orchestra.run.smithi096.stdout:2 2024-09-13T17:41:41.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 msgr2=0x7f4c681ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:41.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 --2- 172.21.15.96:0/92242561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4c58019fc0 tx=0x7f4c5801abd0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 shutdown_connections 2024-09-13T17:41:41.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 --2- 172.21.15.96:0/92242561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c6809dc40 0x7f4c681ca270 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:41.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 >> 172.21.15.96:0/92242561 conn(0x7f4c6806f350 msgr2=0x7f4c68137ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:41.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 shutdown_connections 2024-09-13T17:41:41.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:41.950+0000 7f4c6dada640 1 -- 172.21.15.96:0/92242561 wait complete. 2024-09-13T17:41:41.968 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T17:41:41.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T17:41:42.013 INFO:tasks.ceph.osd.0:Started 2024-09-13T17:41:42.013 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T17:41:42.013 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T17:41:42.015 INFO:tasks.ceph.osd.1:Started 2024-09-13T17:41:42.015 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T17:41:42.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T17:41:42.016 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.014+0000 7f14b9f7d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:41:42.017 INFO:tasks.ceph.osd.2:Started 2024-09-13T17:41:42.017 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46aad88640 1 Processor -- start 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46aad88640 1 -- start start 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46aad88640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f46a4098a60 0x7f46a4098e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46aad88640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a409c020 con 0x7f46a409a810 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46aad88640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a409cc90 con 0x7f46a4098a60 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46a8b24640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f46a4098a60 0x7f46a4098e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46a3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f46a409a810 0x7f46a4134cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39508/0 (socket says 172.21.15.96:39508) 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.126+0000 7f46a3fff640 1 -- 172.21.15.96:0/558052736 learned_addr learned my addr 172.21.15.96:0/558052736 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a8b24640 1 -- 172.21.15.96:0/558052736 >> v1:172.21.15.96:6789/0 conn(0x7f46a409a810 legacy=0x7f46a4134cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a8b24640 1 -- 172.21.15.96:0/558052736 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a409c2e0 con 0x7f46a4098a60 2024-09-13T17:41:42.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a8b24640 1 --2- 172.21.15.96:0/558052736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a4098e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efadfed351b5286 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:42.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a37fe640 1 -- 172.21.15.96:0/558052736 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f469c0a5650 con 0x7f46a4098a60 2024-09-13T17:41:42.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a37fe640 1 -- 172.21.15.96:0/558052736 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c0a57b0 con 0x7f46a4098a60 2024-09-13T17:41:42.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46a37fe640 1 -- 172.21.15.96:0/558052736 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f469c0a5a70 con 0x7f46a4098a60 2024-09-13T17:41:42.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- 172.21.15.96:0/558052736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 msgr2=0x7f46a4098e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 --2- 172.21.15.96:0/558052736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a4098e40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f469c009d90 tx=0x7f469c01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- 172.21.15.96:0/558052736 shutdown_connections 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 --2- 172.21.15.96:0/558052736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a4098e40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- 172.21.15.96:0/558052736 >> 172.21.15.96:0/558052736 conn(0x7f46a406e350 msgr2=0x7f46a4070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- 172.21.15.96:0/558052736 shutdown_connections 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- 172.21.15.96:0/558052736 wait complete. 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 Processor -- start 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.130+0000 7f46aad88640 1 -- start start 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46aad88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46aad88640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a411dc80 con 0x7f46a4098a60 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a8b24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:42.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a8b24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34528/0 (socket says 172.21.15.96:34528) 2024-09-13T17:41:42.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a8b24640 1 -- 172.21.15.96:0/3407309052 learned_addr learned my addr 172.21.15.96:0/3407309052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:42.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a8b24640 1 -- 172.21.15.96:0/3407309052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c009a70 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a8b24640 1 --2- 172.21.15.96:0/3407309052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f469c0194f0 tx=0x7f469c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f469c0b9660 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c0b97c0 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a40a5970 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f469c0b9a60 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a40719f0 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f469c0ad440 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f469c0c2890 con 0x7f46a4098a60 2024-09-13T17:41:42.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.134+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a411e9e0 con 0x7f46a4098a60 2024-09-13T17:41:42.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.138+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f469c0e0e30 con 0x7f46a4098a60 2024-09-13T17:41:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.190+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46a4098e40 con 0x7f46a4098a60 2024-09-13T17:41:42.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.190+0000 7f46a17fa640 1 -- 172.21.15.96:0/3407309052 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3426 (secure 0 0 0) 0x7f469c0cf030 con 0x7f46a4098a60 2024-09-13T17:41:42.191 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:42.191 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:41.949857+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 msgr2=0x7f46a40a3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 --2- 172.21.15.96:0/3407309052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f469c0194f0 tx=0x7f469c019680 comp rx=0 tx=0).stop 2024-09-13T17:41:42.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 shutdown_connections 2024-09-13T17:41:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 --2- 172.21.15.96:0/3407309052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a4098a60 0x7f46a40a3a70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 >> 172.21.15.96:0/3407309052 conn(0x7f46a406e350 msgr2=0x7f46a413a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 shutdown_connections 2024-09-13T17:41:42.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:42.194+0000 7f46aad88640 1 -- 172.21.15.96:0/3407309052 wait complete. 2024-09-13T17:41:42.207 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T17:41:42.207 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T17:41:42.508 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:42.605 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.606+0000 7f14b9f7d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:41:42.617 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-13T17:41:42.618+0000 7fcbea8247c0 -1 Falling back to public interface 2024-09-13T17:41:42.619 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.618+0000 7f66d0ed8640 1 Processor -- start 2024-09-13T17:41:42.619 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.618+0000 7f66d0ed8640 1 -- start start 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66d0ed8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66cc1388d0 0x7f66cc13d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66d0ed8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc06a360 con 0x7f66cc069f80 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66d0ed8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc138cd0 con 0x7f66cc1388d0 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66cc1388d0 0x7f66cc13d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66cc1388d0 0x7f66cc13d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43376/0 (socket says 172.21.15.96:43376) 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66ca59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f66cc069f80 0x7f66cc1381c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40240/0 (socket says 172.21.15.96:40240) 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 -- 172.21.15.96:0/1847291102 learned_addr learned my addr 172.21.15.96:0/1847291102 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 -- 172.21.15.96:0/1847291102 >> v1:172.21.15.96:6789/0 conn(0x7f66cc069f80 legacy=0x7f66cc1381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.621 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 -- 172.21.15.96:0/1847291102 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0a30f0 con 0x7f66cc1388d0 2024-09-13T17:41:42.623 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c9d9b640 1 --2- 172.21.15.96:0/1847291102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc1388d0 0x7f66cc13d0c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66c00097f0 tx=0x7f66c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a97595562785935 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:42.623 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c959a640 1 -- 172.21.15.96:0/1847291102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00ad460 con 0x7f66cc1388d0 2024-09-13T17:41:42.623 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c959a640 1 -- 172.21.15.96:0/1847291102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c00ad5c0 con 0x7f66cc1388d0 2024-09-13T17:41:42.623 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.622+0000 7f66c959a640 1 -- 172.21.15.96:0/1847291102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00ad880 con 0x7f66cc1388d0 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/1847291102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc1388d0 msgr2=0x7f66cc13d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 --2- 172.21.15.96:0/1847291102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc1388d0 0x7f66cc13d0c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f66c00097f0 tx=0x7f66c001c450 comp rx=0 tx=0).stop 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/1847291102 shutdown_connections 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 --2- 172.21.15.96:0/1847291102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc1388d0 0x7f66cc13d0c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/1847291102 >> 172.21.15.96:0/1847291102 conn(0x7f66cc0a75c0 msgr2=0x7f66cc0a79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/1847291102 shutdown_connections 2024-09-13T17:41:42.624 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/1847291102 wait complete. 2024-09-13T17:41:42.625 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 Processor -- start 2024-09-13T17:41:42.625 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- start start 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0a0270 con 0x7f66cc069f80 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43392/0 (socket says 172.21.15.96:43392) 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66ca59c640 1 -- 172.21.15.96:0/3824874658 learned_addr learned my addr 172.21.15.96:0/3824874658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66ca59c640 1 -- 172.21.15.96:0/3824874658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c00094d0 con 0x7f66cc069f80 2024-09-13T17:41:42.626 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66ca59c640 1 --2- 172.21.15.96:0/3824874658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00c3b0 tx=0x7f66bc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66bc007410 con 0x7f66cc069f80 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc007570 con 0x7f66cc069f80 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc139960 con 0x7f66cc069f80 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66bc0047f0 con 0x7f66cc069f80 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc139c70 con 0x7f66cc069f80 2024-09-13T17:41:42.627 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) v1 ==== 668+0+0 (secure 0 0 0) 0x7f66bc004c50 con 0x7f66cc069f80 2024-09-13T17:41:42.628 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f66bc013890 con 0x7f66cc069f80 2024-09-13T17:41:42.628 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.626+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc071160 con 0x7f66cc069f80 2024-09-13T17:41:42.629 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.630+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73245 (secure 0 0 0) 0x7f66bc013a50 con 0x7f66cc069f80 2024-09-13T17:41:42.632 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-13T17:41:42.634+0000 7ff82941b7c0 -1 Falling back to public interface 2024-09-13T17:41:42.652 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-13T17:41:42.654+0000 7fa5260577c0 -1 Falling back to public interface 2024-09-13T17:41:42.686 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.686+0000 7f14b9f7d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:41:42.713 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66cc1381c0 con 0x7f66cc069f80 2024-09-13T17:41:42.713 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66baffd640 1 -- 172.21.15.96:0/3824874658 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v5) v1 ==== 74+0+3426 (secure 0 0 0) 0x7f66bc0134b0 con 0x7f66cc069f80 2024-09-13T17:41:42.713 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-13T17:41:42.713 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":5,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:41.949857+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:42.714 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 msgr2=0x7f66cc13a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:42.714 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 --2- 172.21.15.96:0/3824874658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00c3b0 tx=0x7f66bc00c870 comp rx=0 tx=0).stop 2024-09-13T17:41:42.714 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 shutdown_connections 2024-09-13T17:41:42.715 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 --2- 172.21.15.96:0/3824874658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66cc069f80 0x7f66cc13a3c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:42.715 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 >> 172.21.15.96:0/3824874658 conn(0x7f66cc0a75c0 msgr2=0x7f66cc134e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:42.715 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 shutdown_connections 2024-09-13T17:41:42.715 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:42.714+0000 7f66d0ed8640 1 -- 172.21.15.96:0/3824874658 wait complete. 2024-09-13T17:41:42.727 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-13T17:41:42.745 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.746+0000 7f14b9f7d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:41:42.828 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.830+0000 7f14b9f7d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:41:42.885 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.886+0000 7f14b9f7d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:41:42.965 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:42.966+0000 7f14b9f7d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:41:43.017 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:43.018+0000 7f14b9f7d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:41:43.168 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:43.166+0000 7f14b9f7d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:41:43.227 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:43.226+0000 7f14b9f7d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:41:43.866 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:41:43.866 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:41:43.866 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:41:43.926 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:43.926+0000 7f14b9f7d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:41:44.096 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:44.094+0000 7f14b9f7d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:41:44.254 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:44.254+0000 7f14b9f7d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:41:44.317 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:44.318+0000 7f14b9f7d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:41:44.449 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:44.450+0000 7f14b9f7d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:41:45.056 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.058+0000 7f14b9f7d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:41:45.132 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.134+0000 7f14b9f7d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:41:45.314 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.314+0000 7f14b9f7d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:41:45.388 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.386+0000 7f14b9f7d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:41:45.699 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.698+0000 7f14b9f7d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:41:45.763 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:45.762+0000 7f14b9f7d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:41:46.252 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-13T17:41:46.254+0000 7ff82941b7c0 -1 osd.0 0 log_to_monitors true 2024-09-13T17:41:46.302 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-13T17:41:46.302+0000 7fcbea8247c0 -1 osd.2 0 log_to_monitors true 2024-09-13T17:41:46.526 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-13T17:41:46.526+0000 7fa5260577c0 -1 osd.1 0 log_to_monitors true 2024-09-13T17:41:46.799 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:41:46.798+0000 7f1461532640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-13T17:41:49.031 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:49.190 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc07c09d640 1 Processor -- start 2024-09-13T17:41:49.190 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc07c09d640 1 -- start start 2024-09-13T17:41:49.191 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc07c09d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc074137f40 0x7fc074138320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:49.191 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc07c09d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07409a180 con 0x7fc07409bd40 2024-09-13T17:41:49.191 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc07c09d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07409a3b0 con 0x7fc074137f40 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc079638640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc07409bd40 0x7fc074136ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40248/0 (socket says 172.21.15.96:40248) 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc079e39640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc074137f40 0x7fc074138320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc079638640 1 -- 172.21.15.96:0/2299848330 learned_addr learned my addr 172.21.15.96:0/2299848330 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc078e37640 1 -- 172.21.15.96:0/2299848330 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111504987 0 0) 0x7fc07409a180 con 0x7fc07409bd40 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc078e37640 1 -- 172.21.15.96:0/2299848330 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc060003530 con 0x7fc07409bd40 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc079e39640 1 -- 172.21.15.96:0/2299848330 >> v1:172.21.15.96:6789/0 conn(0x7fc07409bd40 legacy=0x7fc074136ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:49.192 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.190+0000 7fc079e39640 1 -- 172.21.15.96:0/2299848330 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07409bb70 con 0x7fc074137f40 2024-09-13T17:41:49.195 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.194+0000 7fc079e39640 1 --2- 172.21.15.96:0/2299848330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074137f40 0x7fc074138320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc070009740 tx=0x7fc07001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2decf1e88648991d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:49.195 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.194+0000 7fc078e37640 1 -- 172.21.15.96:0/2299848330 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0700ab530 con 0x7fc074137f40 2024-09-13T17:41:49.195 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.194+0000 7fc078e37640 1 -- 172.21.15.96:0/2299848330 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0700a7050 con 0x7fc074137f40 2024-09-13T17:41:49.196 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.194+0000 7fc078e37640 1 -- 172.21.15.96:0/2299848330 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0700ab7e0 con 0x7fc074137f40 2024-09-13T17:41:49.197 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- 172.21.15.96:0/2299848330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074137f40 msgr2=0x7fc074138320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:49.197 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 --2- 172.21.15.96:0/2299848330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074137f40 0x7fc074138320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc070009740 tx=0x7fc07001c390 comp rx=0 tx=0).stop 2024-09-13T17:41:49.197 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- 172.21.15.96:0/2299848330 shutdown_connections 2024-09-13T17:41:49.197 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 --2- 172.21.15.96:0/2299848330 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074137f40 0x7fc074138320 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc070009740 tx=0x7fc07001c390 comp rx=0 tx=0).stop 2024-09-13T17:41:49.197 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- 172.21.15.96:0/2299848330 >> 172.21.15.96:0/2299848330 conn(0x7fc07406cc10 msgr2=0x7fc074098a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:49.198 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- 172.21.15.96:0/2299848330 shutdown_connections 2024-09-13T17:41:49.198 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- 172.21.15.96:0/2299848330 wait complete. 2024-09-13T17:41:49.198 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 Processor -- start 2024-09-13T17:41:49.198 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- start start 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc07c09d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740a91c0 con 0x7fc07409bd40 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc079e39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc079e39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43512/0 (socket says 172.21.15.96:43512) 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.198+0000 7fc079e39640 1 -- 172.21.15.96:0/1789510760 learned_addr learned my addr 172.21.15.96:0/1789510760 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:49.200 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc079e39640 1 -- 172.21.15.96:0/1789510760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070009420 con 0x7fc07409bd40 2024-09-13T17:41:49.201 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc079e39640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc070019fc0 tx=0x7fc07001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:49.202 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc070006e40 con 0x7fc07409bd40 2024-09-13T17:41:49.202 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07409fcd0 con 0x7fc07409bd40 2024-09-13T17:41:49.202 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0740a94d0 con 0x7fc07409bd40 2024-09-13T17:41:49.202 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0700b7450 con 0x7fc07409bd40 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc0700b76f0 con 0x7fc07409bd40 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.202+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c005260 con 0x7fc07409bd40 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.206+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) v1 ==== 43251+0+0 (secure 0 0 0) 0x7fc0700b78d0 con 0x7fc07409bd40 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.206+0000 7fc06a7fc640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 0x7fc0580378b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.206+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1361+0+0 (secure 0 0 0) 0x7fc0700e1a00 con 0x7fc07409bd40 2024-09-13T17:41:49.204 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.206+0000 7fc079638640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 0x7fc0580378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:49.205 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.206+0000 7fc079638640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 0x7fc0580378b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc064007fd0 tx=0x7fc064007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:49.213 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.214+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc0700c76c0 con 0x7fc07409bd40 2024-09-13T17:41:49.494 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.494+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc05c005510 con 0x7fc07409bd40 2024-09-13T17:41:49.495 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.494+0000 7fc06a7fc640 1 -- 172.21.15.96:0/1789510760 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v7) v1 ==== 74+0+3426 (secure 0 0 0) 0x7fc0700b0070 con 0x7fc07409bd40 2024-09-13T17:41:49.495 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-13T17:41:49.495 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":7,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:47.847701+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:49.497 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 msgr2=0x7fc0580378b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:49.497 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 0x7fc0580378b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc064007fd0 tx=0x7fc064007980 comp rx=0 tx=0).stop 2024-09-13T17:41:49.497 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 msgr2=0x7fc0741cc9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:49.498 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc070019fc0 tx=0x7fc07001abd0 comp rx=0 tx=0).stop 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 shutdown_connections 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc0580353f0 0x7fc0580378b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 --2- 172.21.15.96:0/1789510760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc07409bd40 0x7fc0741cc9a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 >> 172.21.15.96:0/1789510760 conn(0x7fc07406cc10 msgr2=0x7fc074139010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 shutdown_connections 2024-09-13T17:41:49.499 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:49.498+0000 7fc07c09d640 1 -- 172.21.15.96:0/1789510760 wait complete. 2024-09-13T17:41:49.520 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-13T17:41:49.585 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-13T17:41:49.586+0000 7fcbe5fa6640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T17:41:49.625 INFO:tasks.ceph.osd.2.smithi096.stderr:2024-09-13T17:41:49.626+0000 7fcbde5eb640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:49.644 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-13T17:41:49.646+0000 7ff824b9d640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T17:41:49.664 INFO:tasks.ceph.osd.0.smithi096.stderr:2024-09-13T17:41:49.666+0000 7ff81d1e2640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:49.678 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-13T17:41:49.678+0000 7fa5217d9640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T17:41:49.701 INFO:tasks.ceph.osd.1.smithi096.stderr:2024-09-13T17:41:49.702+0000 7fa519e1e640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T17:41:55.823 DEBUG:teuthology.orchestra.run.smithi096:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:55.944 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc1340f640 1 Processor -- start 2024-09-13T17:41:55.944 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc1340f640 1 -- start start 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc1340f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc0c06f760 0x7efc0c09d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc1340f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc0c09d100 con 0x7efc0c070f00 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc1340f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0c070d50 con 0x7efc0c06f760 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc111ab640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc0c06f760 0x7efc0c09d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc109aa640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efc0c070f00 0x7efc0c06fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43202/0 (socket says 172.21.15.96:43202) 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc109aa640 1 -- 172.21.15.96:0/2600498853 learned_addr learned my addr 172.21.15.96:0/2600498853 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efbfffff640 1 -- 172.21.15.96:0/2600498853 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 639453874 0 0) 0x7efc0c09d100 con 0x7efc0c070f00 2024-09-13T17:41:55.945 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efbfffff640 1 -- 172.21.15.96:0/2600498853 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbf8003530 con 0x7efc0c070f00 2024-09-13T17:41:55.946 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc111ab640 1 -- 172.21.15.96:0/2600498853 >> v1:172.21.15.96:6789/0 conn(0x7efc0c070f00 legacy=0x7efc0c06fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:55.946 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.946+0000 7efc111ab640 1 -- 172.21.15.96:0/2600498853 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0c0a67f0 con 0x7efc0c06f760 2024-09-13T17:41:55.949 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 --2- 172.21.15.96:0/2600498853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c09d850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc08009740 tx=0x7efc0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd324e0fee80967a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:55.949 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efbfffff640 1 -- 172.21.15.96:0/2600498853 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc080ab530 con 0x7efc0c06f760 2024-09-13T17:41:55.949 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efbfffff640 1 -- 172.21.15.96:0/2600498853 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc080a7050 con 0x7efc0c06f760 2024-09-13T17:41:55.949 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efbfffff640 1 -- 172.21.15.96:0/2600498853 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc080ab7e0 con 0x7efc0c06f760 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- 172.21.15.96:0/2600498853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 msgr2=0x7efc0c09d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 --2- 172.21.15.96:0/2600498853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c09d850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc08009740 tx=0x7efc0801c420 comp rx=0 tx=0).stop 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- 172.21.15.96:0/2600498853 shutdown_connections 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 --2- 172.21.15.96:0/2600498853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c09d850 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- 172.21.15.96:0/2600498853 >> 172.21.15.96:0/2600498853 conn(0x7efc0c098e60 msgr2=0x7efc0c09b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- 172.21.15.96:0/2600498853 shutdown_connections 2024-09-13T17:41:55.950 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- 172.21.15.96:0/2600498853 wait complete. 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 Processor -- start 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- start start 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc1340f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0c0a2f60 con 0x7efc0c06f760 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34912/0 (socket says 172.21.15.96:34912) 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 -- 172.21.15.96:0/3316116010 learned_addr learned my addr 172.21.15.96:0/3316116010 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:55.952 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 -- 172.21.15.96:0/3316116010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08009420 con 0x7efc0c06f760 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.950+0000 7efc111ab640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc080ae040 tx=0x7efc080191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc080abe10 con 0x7efc0c06f760 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc080b4410 con 0x7efc0c06f760 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc080b46d0 con 0x7efc0c06f760 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0c0a4fe0 con 0x7efc0c06f760 2024-09-13T17:41:55.953 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc0c137f40 con 0x7efc0c06f760 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0c0a4e30 con 0x7efc0c06f760 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7efc080b2040 con 0x7efc0c06f760 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 0x7efbf0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7efc080e1380 con 0x7efc0c06f760 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efc109aa640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 0x7efbf0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:55.954 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.954+0000 7efc109aa640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 0x7efbf0037910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc00007fd0 tx=0x7efc00007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:55.958 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:55.958+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc080bc3b0 con 0x7efc0c06f760 2024-09-13T17:41:56.062 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:41:56.117 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.118+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc0c139250 con 0x7efc0c06f760 2024-09-13T17:41:56.118 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.118+0000 7efbfdffb640 1 -- 172.21.15.96:0/3316116010 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6720 (secure 0 0 0) 0x7efc080b0030 con 0x7efc0c06f760 2024-09-13T17:41:56.119 INFO:teuthology.misc.health.smithi096.stdout: 2024-09-13T17:41:56.119 INFO:teuthology.misc.health.smithi096.stdout:{"epoch":11,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:53.898379+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 msgr2=0x7efbf0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 0x7efbf0037910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc00007fd0 tx=0x7efc00007980 comp rx=0 tx=0).stop 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 msgr2=0x7efc0c0a5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efc080ae040 tx=0x7efc080191c0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 shutdown_connections 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efbf0035450 0x7efbf0037910 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 --2- 172.21.15.96:0/3316116010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc0c06f760 0x7efc0c0a5320 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 >> 172.21.15.96:0/3316116010 conn(0x7efc0c098e60 msgr2=0x7efc0c1352a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.121 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 shutdown_connections 2024-09-13T17:41:56.122 INFO:teuthology.misc.health.smithi096.stderr:2024-09-13T17:41:56.122+0000 7efc1340f640 1 -- 172.21.15.96:0/3316116010 wait complete. 2024-09-13T17:41:56.137 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-13T17:41:56.137 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T17:41:56.137 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:41:56.137 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:41:56.181 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:41:56.182 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:41:56.182 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:41:56.184 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:41:56.185 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-13T17:41:56.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:56.263 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:41:56.264 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:41:56.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e93c0640 1 Processor -- start 2024-09-13T17:41:56.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e93c0640 1 -- start start 2024-09-13T17:41:56.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e93c0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23e413af50 0x7f23e413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e93c0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e4138050 con 0x7f23e409d690 2024-09-13T17:41:56.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e93c0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e409c410 con 0x7f23e413af50 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e27fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23e413af50 0x7f23e413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e27fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23e413af50 0x7f23e413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34958/0 (socket says 172.21.15.96:34958) 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e27fc640 1 -- 172.21.15.96:0/2362572105 learned_addr learned my addr 172.21.15.96:0/2362572105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e2ffd640 1 --1- 172.21.15.96:0/2362572105 >> v1:172.21.15.96:6789/0 conn(0x7f23e409d690 0x7f23e4138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43228/0 (socket says 172.21.15.96:43228) 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e27fc640 1 -- 172.21.15.96:0/2362572105 >> v1:172.21.15.96:6789/0 conn(0x7f23e409d690 legacy=0x7f23e4138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2362572105 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286773178 0 0) 0x7f23d4001120 con 0x7f23e409d690 2024-09-13T17:41:56.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.310+0000 7f23e27fc640 1 -- 172.21.15.96:0/2362572105 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e409cba0 con 0x7f23e413af50 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e27fc640 1 --2- 172.21.15.96:0/2362572105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e413af50 0x7f23e413d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23d80097f0 tx=0x7f23d801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe66ce33a57b8c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2362572105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d80a8900 con 0x7f23e413af50 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2362572105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d80a8a60 con 0x7f23e413af50 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e1ffb640 1 -- 172.21.15.96:0/2362572105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d80a8d20 con 0x7f23e413af50 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/2362572105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e413af50 msgr2=0x7f23e413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 --2- 172.21.15.96:0/2362572105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e413af50 0x7f23e413d340 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f23d80097f0 tx=0x7f23d801ecd0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/2362572105 shutdown_connections 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 --2- 172.21.15.96:0/2362572105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e413af50 0x7f23e413d340 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/2362572105 >> 172.21.15.96:0/2362572105 conn(0x7f23e406bc60 msgr2=0x7f23e406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/2362572105 shutdown_connections 2024-09-13T17:41:56.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/2362572105 wait complete. 2024-09-13T17:41:56.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 Processor -- start 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- start start 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e40a6280 con 0x7f23e409d690 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34974/0 (socket says 172.21.15.96:34974) 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e2ffd640 1 -- 172.21.15.96:0/3499245808 learned_addr learned my addr 172.21.15.96:0/3499245808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e2ffd640 1 -- 172.21.15.96:0/3499245808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d80094d0 con 0x7f23e409d690 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e2ffd640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c390 tx=0x7f23d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d4007410 con 0x7f23e409d690 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d4007590 con 0x7f23e409d690 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e40a51d0 con 0x7f23e409d690 2024-09-13T17:41:56.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d40047f0 con 0x7f23e409d690 2024-09-13T17:41:56.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.314+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e40a7170 con 0x7f23e409d690 2024-09-13T17:41:56.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f23d4004950 con 0x7f23e409d690 2024-09-13T17:41:56.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23cb7fe640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 0x7f23d0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e40a5ee0 con 0x7f23e409d690 2024-09-13T17:41:56.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f23d4046f40 con 0x7f23e409d690 2024-09-13T17:41:56.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23e27fc640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 0x7f23d0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.318+0000 7f23e27fc640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 0x7f23d0037910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23d80b2000 tx=0x7f23d80aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.322+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23d401bbc0 con 0x7f23e409d690 2024-09-13T17:41:56.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.478+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23e409bc00 con 0x7f23e409d690 2024-09-13T17:41:56.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23cb7fe640 1 -- 172.21.15.96:0/3499245808 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6720 (secure 0 0 0) 0x7f23e409bc00 con 0x7f23e409d690 2024-09-13T17:41:56.481 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:56.483 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:53.898379+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:56.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 msgr2=0x7f23d0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 0x7f23d0037910 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23d80b2000 tx=0x7f23d80aa070 comp rx=0 tx=0).stop 2024-09-13T17:41:56.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 msgr2=0x7f23e40a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c390 tx=0x7f23d400c850 comp rx=0 tx=0).stop 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 shutdown_connections 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f23d0035450 0x7f23d0037910 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 --2- 172.21.15.96:0/3499245808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e409d690 0x7f23e40a6c30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 >> 172.21.15.96:0/3499245808 conn(0x7f23e406bc60 msgr2=0x7f23e4139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 shutdown_connections 2024-09-13T17:41:56.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.482+0000 7f23e93c0640 1 -- 172.21.15.96:0/3499245808 wait complete. 2024-09-13T17:41:56.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:41:56.499 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:41:56.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.610+0000 7f1d4a7b8640 1 Processor -- start 2024-09-13T17:41:56.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.610+0000 7f1d4a7b8640 1 -- start start 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d4a7b8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1d4406f6d0 0x7f1d44134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d4a7b8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4406e130 con 0x7f1d4409dc20 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d4a7b8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d440a6750 con 0x7f1d4406f6d0 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d43fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1d4406f6d0 0x7f1d44134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d43fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1d4406f6d0 0x7f1d44134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34976/0 (socket says 172.21.15.96:34976) 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d437fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1d4409dc20 0x7f1d44134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43240/0 (socket says 172.21.15.96:43240) 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d43fff640 1 -- 172.21.15.96:0/712918118 learned_addr learned my addr 172.21.15.96:0/712918118 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d43fff640 1 -- 172.21.15.96:0/712918118 >> v1:172.21.15.96:6789/0 conn(0x7f1d4409dc20 legacy=0x7f1d44134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d42ffd640 1 -- 172.21.15.96:0/712918118 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943885914 0 0) 0x7f1d4406e130 con 0x7f1d4409dc20 2024-09-13T17:41:56.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.614+0000 7f1d43fff640 1 -- 172.21.15.96:0/712918118 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440712e0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d43fff640 1 --2- 172.21.15.96:0/712918118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400d150 tx=0x7f1d3401eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7be1bed8541f887 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d42ffd640 1 -- 172.21.15.96:0/712918118 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d3400b0d0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d42ffd640 1 -- 172.21.15.96:0/712918118 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3400b230 con 0x7f1d4406f6d0 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d42ffd640 1 -- 172.21.15.96:0/712918118 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d340a7810 con 0x7f1d4406f6d0 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/712918118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 msgr2=0x7f1d44134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/712918118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134ab0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1d3400d150 tx=0x7f1d3401eba0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/712918118 shutdown_connections 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/712918118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134ab0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/712918118 >> 172.21.15.96:0/712918118 conn(0x7f1d44098a60 msgr2=0x7f1d4409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/712918118 shutdown_connections 2024-09-13T17:41:56.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.618+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/712918118 wait complete. 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 Processor -- start 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 -- start start 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4411ebd0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34986/0 (socket says 172.21.15.96:34986) 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d43fff640 1 -- 172.21.15.96:0/3064883360 learned_addr learned my addr 172.21.15.96:0/3064883360 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d43fff640 1 -- 172.21.15.96:0/3064883360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3400bd40 con 0x7f1d4406f6d0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d43fff640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d3401b7b0 tx=0x7f1d3401b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d34009600 con 0x7f1d4406f6d0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34009760 con 0x7f1d4406f6d0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4411da30 con 0x7f1d4406f6d0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d34009a00 con 0x7f1d4406f6d0 2024-09-13T17:41:56.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4411dcd0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4406f310 con 0x7f1d4406f6d0 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f1d34009b60 con 0x7f1d4406f6d0 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 0x7f1d30037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f1d340e4be0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d437fe640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 0x7f1d30037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.622+0000 7f1d437fe640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 0x7f1d30037910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007fd0 tx=0x7f1d38007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.626+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1d340b73a0 con 0x7f1d4406f6d0 2024-09-13T17:41:56.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.774+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d4411cd50 con 0x7f1d4406f6d0 2024-09-13T17:41:56.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.774+0000 7f1d40ff9640 1 -- 172.21.15.96:0/3064883360 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v11) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d340be070 con 0x7f1d4406f6d0 2024-09-13T17:41:56.775 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 msgr2=0x7f1d30037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 0x7f1d30037910 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007fd0 tx=0x7f1d38007920 comp rx=0 tx=0).stop 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 msgr2=0x7f1d44134970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1d3401b7b0 tx=0x7f1d3401b940 comp rx=0 tx=0).stop 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 shutdown_connections 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1d30035450 0x7f1d30037910 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 --2- 172.21.15.96:0/3064883360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d4406f6d0 0x7f1d44134970 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 >> 172.21.15.96:0/3064883360 conn(0x7f1d44098a60 msgr2=0x7f1d44098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 shutdown_connections 2024-09-13T17:41:56.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.778+0000 7f1d4a7b8640 1 -- 172.21.15.96:0/3064883360 wait complete. 2024-09-13T17:41:56.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:41:56.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.910+0000 7f589468e640 1 Processor -- start 2024-09-13T17:41:56.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.910+0000 7f589468e640 1 -- start start 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f588c136170 0x7f588c136550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f588c135cd0 con 0x7f588c09a0a0 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c099c90 con 0x7f588c136170 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f588c136170 0x7f588c136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f588c136170 0x7f588c136550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34994/0 (socket says 172.21.15.96:34994) 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f5891c29640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f588c09a0a0 0x7f588c132650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43248/0 (socket says 172.21.15.96:43248) 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 -- 172.21.15.96:0/3362398509 learned_addr learned my addr 172.21.15.96:0/3362398509 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 -- 172.21.15.96:0/3362398509 >> v1:172.21.15.96:6789/0 conn(0x7f588c09a0a0 legacy=0x7f588c132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 -- 172.21.15.96:0/3362398509 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c09d980 con 0x7f588c136170 2024-09-13T17:41:56.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f5891428640 1 -- 172.21.15.96:0/3362398509 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388748493 0 0) 0x7f588c135cd0 con 0x7f588c09a0a0 2024-09-13T17:41:56.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589242a640 1 --2- 172.21.15.96:0/3362398509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c136170 0x7f588c136550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58880097f0 tx=0x7f588801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150618c932729df7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f5891428640 1 -- 172.21.15.96:0/3362398509 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f588c133570 con 0x7f588c136170 2024-09-13T17:41:56.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f5891428640 1 -- 172.21.15.96:0/3362398509 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58880a7050 con 0x7f588c136170 2024-09-13T17:41:56.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f5891428640 1 -- 172.21.15.96:0/3362398509 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58880ab680 con 0x7f588c136170 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- 172.21.15.96:0/3362398509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c136170 msgr2=0x7f588c136550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 --2- 172.21.15.96:0/3362398509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c136170 0x7f588c136550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f58880097f0 tx=0x7f588801c450 comp rx=0 tx=0).stop 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- 172.21.15.96:0/3362398509 shutdown_connections 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 --2- 172.21.15.96:0/3362398509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c136170 0x7f588c136550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- 172.21.15.96:0/3362398509 >> 172.21.15.96:0/3362398509 conn(0x7f588c09c210 msgr2=0x7f588c071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- 172.21.15.96:0/3362398509 shutdown_connections 2024-09-13T17:41:56.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.914+0000 7f589468e640 1 -- 172.21.15.96:0/3362398509 wait complete. 2024-09-13T17:41:56.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 Processor -- start 2024-09-13T17:41:56.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 -- start start 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1395b0 con 0x7f588c09a0a0 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589242a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589242a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35002/0 (socket says 172.21.15.96:35002) 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589242a640 1 -- 172.21.15.96:0/3165881314 learned_addr learned my addr 172.21.15.96:0/3165881314 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589242a640 1 -- 172.21.15.96:0/3165881314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880094d0 con 0x7f588c09a0a0 2024-09-13T17:41:56.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589242a640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5888019060 tx=0x7f588801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5888004020 con 0x7f588c09a0a0 2024-09-13T17:41:56.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5888004180 con 0x7f588c09a0a0 2024-09-13T17:41:56.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5888004420 con 0x7f588c09a0a0 2024-09-13T17:41:56.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c13bac0 con 0x7f588c09a0a0 2024-09-13T17:41:56.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c13b7b0 con 0x7f588c09a0a0 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f58880b2040 con 0x7f588c09a0a0 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c139c50 con 0x7f588c09a0a0 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 0x7f58700378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f58880e14a0 con 0x7f588c09a0a0 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5891c29640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 0x7f58700378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:56.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.918+0000 7f5891c29640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 0x7f58700378c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:56.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:56.926+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58880c68f0 con 0x7f588c09a0a0 2024-09-13T17:41:57.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.114+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f588c099a70 con 0x7f588c09a0a0 2024-09-13T17:41:57.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f5882ffd640 1 -- 172.21.15.96:0/3165881314 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1250 (secure 0 0 0) 0x7f58880b0070 con 0x7f588c09a0a0 2024-09-13T17:41:57.116 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:57.117 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:41:57.117 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 2 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 msgr2=0x7f58700378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 0x7f58700378c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f587c007fd0 tx=0x7f587c007920 comp rx=0 tx=0).stop 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 msgr2=0x7f588c09f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5888019060 tx=0x7f588801ac90 comp rx=0 tx=0).stop 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 shutdown_connections 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5870035400 0x7f58700378c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:57.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 --2- 172.21.15.96:0/3165881314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f588c09a0a0 0x7f588c09f6b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 >> 172.21.15.96:0/3165881314 conn(0x7f588c09c210 msgr2=0x7f588c12c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 shutdown_connections 2024-09-13T17:41:57.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:57.118+0000 7f589468e640 1 -- 172.21.15.96:0/3165881314 wait complete. 2024-09-13T17:41:59.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:41:59.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2889c84640 1 Processor -- start 2024-09-13T17:41:59.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2889c84640 1 -- start start 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2889c84640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f288409e820 0x7f288413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2889c84640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840a68b0 con 0x7f2884134e70 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2889c84640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288409ddc0 con 0x7f288409e820 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2882ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f288409e820 0x7f288413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f28837fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2884134e70 0x7f2884135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43262/0 (socket says 172.21.15.96:43262) 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.266+0000 7f2882ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f288409e820 0x7f288413ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35006/0 (socket says 172.21.15.96:35006) 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f28837fe640 1 -- 172.21.15.96:0/1563585538 learned_addr learned my addr 172.21.15.96:0/1563585538 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2882ffd640 1 -- 172.21.15.96:0/1563585538 >> v1:172.21.15.96:6789/0 conn(0x7f2884134e70 legacy=0x7f2884135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2882ffd640 1 -- 172.21.15.96:0/1563585538 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884136970 con 0x7f288409e820 2024-09-13T17:41:59.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2882ffd640 1 --2- 172.21.15.96:0/1563585538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288413ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2874009740 tx=0x7f287401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ed5ff010c032d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f28827fc640 1 -- 172.21.15.96:0/1563585538 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28740ab530 con 0x7f288409e820 2024-09-13T17:41:59.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f28827fc640 1 -- 172.21.15.96:0/1563585538 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28740a7050 con 0x7f288409e820 2024-09-13T17:41:59.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f28827fc640 1 -- 172.21.15.96:0/1563585538 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28740ab7e0 con 0x7f288409e820 2024-09-13T17:41:59.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 -- 172.21.15.96:0/1563585538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 msgr2=0x7f288413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 --2- 172.21.15.96:0/1563585538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288413ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2874009740 tx=0x7f287401c420 comp rx=0 tx=0).stop 2024-09-13T17:41:59.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 -- 172.21.15.96:0/1563585538 shutdown_connections 2024-09-13T17:41:59.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 --2- 172.21.15.96:0/1563585538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288413ae50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 -- 172.21.15.96:0/1563585538 >> 172.21.15.96:0/1563585538 conn(0x7f2884098a60 msgr2=0x7f288409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 -- 172.21.15.96:0/1563585538 shutdown_connections 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.270+0000 7f2889c84640 1 -- 172.21.15.96:0/1563585538 wait complete. 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 Processor -- start 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 -- start start 2024-09-13T17:41:59.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840a50e0 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f28837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f28837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35016/0 (socket says 172.21.15.96:35016) 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f28837fe640 1 -- 172.21.15.96:0/1950614867 learned_addr learned my addr 172.21.15.96:0/1950614867 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f28837fe640 1 -- 172.21.15.96:0/1950614867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009420 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f28837fe640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f287800c480 tx=0x7f287800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2878005c20 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2878005d80 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840a3fd0 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28780071a0 con 0x7f288409e820 2024-09-13T17:41:59.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2884071600 con 0x7f288409e820 2024-09-13T17:41:59.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2878007300 con 0x7f288409e820 2024-09-13T17:41:59.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28840008e0 con 0x7f288409e820 2024-09-13T17:41:59.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 0x7f28700378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f2878046830 con 0x7f288409e820 2024-09-13T17:41:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2882ffd640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 0x7f28700378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.274+0000 7f2882ffd640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 0x7f28700378c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28740ad000 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.282+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2878012400 con 0x7f288409e820 2024-09-13T17:41:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.466+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28840a4470 con 0x7f288409e820 2024-09-13T17:41:59.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.466+0000 7f286bfff640 1 -- 172.21.15.96:0/1950614867 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2006 (secure 0 0 0) 0x7f2878017020 con 0x7f288409e820 2024-09-13T17:41:59.467 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:59.468 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4145,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2024-09-13T17:41:59.468 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-09-13T17:41:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 msgr2=0x7f28700378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 0x7f28700378c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f28740ad000 tx=0x7f2874008040 comp rx=0 tx=0).stop 2024-09-13T17:41:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 msgr2=0x7f288412f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f287800c480 tx=0x7f287800c940 comp rx=0 tx=0).stop 2024-09-13T17:41:59.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 shutdown_connections 2024-09-13T17:41:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2870035400 0x7f28700378c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 --2- 172.21.15.96:0/1950614867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f288409e820 0x7f288412f5e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 >> 172.21.15.96:0/1950614867 conn(0x7f2884098a60 msgr2=0x7f2884098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 shutdown_connections 2024-09-13T17:41:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.470+0000 7f2889c84640 1 -- 172.21.15.96:0/1950614867 wait complete. 2024-09-13T17:41:59.484 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:41:59.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.602+0000 7f8180569640 1 Processor -- start 2024-09-13T17:41:59.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.602+0000 7f8180569640 1 -- start start 2024-09-13T17:41:59.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f8180569640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f81781366a0 0x7f817813ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f8180569640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8178136ab0 con 0x7f81780a0860 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f8180569640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8178071cd0 con 0x7f81781366a0 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817db04640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f81781366a0 0x7f817813ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817e305640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f81780a0860 0x7f8178135f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43278/0 (socket says 172.21.15.96:43278) 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817e305640 1 -- 172.21.15.96:0/2928959279 learned_addr learned my addr 172.21.15.96:0/2928959279 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817d303640 1 -- 172.21.15.96:0/2928959279 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898652542 0 0) 0x7f8178136ab0 con 0x7f81780a0860 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817d303640 1 -- 172.21.15.96:0/2928959279 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8164003530 con 0x7f81780a0860 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817db04640 1 -- 172.21.15.96:0/2928959279 >> v1:172.21.15.96:6789/0 conn(0x7f81780a0860 legacy=0x7f8178135f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817db04640 1 -- 172.21.15.96:0/2928959279 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81780a0c40 con 0x7f81781366a0 2024-09-13T17:41:59.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817db04640 1 --2- 172.21.15.96:0/2928959279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81781366a0 0x7f817813ae90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8168009740 tx=0x7f816801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8baba9813791653 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817d303640 1 -- 172.21.15.96:0/2928959279 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81680ab530 con 0x7f81781366a0 2024-09-13T17:41:59.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817d303640 1 -- 172.21.15.96:0/2928959279 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81680a7050 con 0x7f81781366a0 2024-09-13T17:41:59.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.606+0000 7f817d303640 1 -- 172.21.15.96:0/2928959279 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81680ab7e0 con 0x7f81781366a0 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2928959279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81781366a0 msgr2=0x7f817813ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 --2- 172.21.15.96:0/2928959279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81781366a0 0x7f817813ae90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8168009740 tx=0x7f816801c390 comp rx=0 tx=0).stop 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2928959279 shutdown_connections 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 --2- 172.21.15.96:0/2928959279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81781366a0 0x7f817813ae90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2928959279 >> 172.21.15.96:0/2928959279 conn(0x7f81780a7390 msgr2=0x7f81780a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2928959279 shutdown_connections 2024-09-13T17:41:59.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2928959279 wait complete. 2024-09-13T17:41:59.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 Processor -- start 2024-09-13T17:41:59.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- start start 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178135880 con 0x7f81780a0860 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f817e305640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f817e305640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35032/0 (socket says 172.21.15.96:35032) 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f817e305640 1 -- 172.21.15.96:0/2515337477 learned_addr learned my addr 172.21.15.96:0/2515337477 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:59.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f817e305640 1 -- 172.21.15.96:0/2515337477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8168009420 con 0x7f81780a0860 2024-09-13T17:41:59.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f817e305640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f817400c450 tx=0x7f817400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8174005c20 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174005d80 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81740071a0 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81780a02f0 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.610+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81780a0590 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8174007300 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178071880 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f816effd640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 0x7f815c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f8174046820 con 0x7f81780a0860 2024-09-13T17:41:59.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f817db04640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 0x7f815c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.614+0000 7f817db04640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 0x7f815c037910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81680ad000 tx=0x7f8168008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.618+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8174012400 con 0x7f81780a0860 2024-09-13T17:41:59.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.770+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8178135f90 con 0x7f81780a0860 2024-09-13T17:41:59.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.770+0000 7f816effd640 1 -- 172.21.15.96:0/2515337477 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+6720 (secure 0 0 0) 0x7f8174017030 con 0x7f81780a0860 2024-09-13T17:41:59.770 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:41:59.770 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:41:53.898379+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:41:59.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 msgr2=0x7f815c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 0x7f815c037910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81680ad000 tx=0x7f8168008040 comp rx=0 tx=0).stop 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 msgr2=0x7f817809e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f817400c450 tx=0x7f817400c910 comp rx=0 tx=0).stop 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 shutdown_connections 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f815c035450 0x7f815c037910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 --2- 172.21.15.96:0/2515337477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81780a0860 0x7f817809e650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 >> 172.21.15.96:0/2515337477 conn(0x7f81780a7390 msgr2=0x7f8178132b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 shutdown_connections 2024-09-13T17:41:59.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.774+0000 7f8180569640 1 -- 172.21.15.96:0/2515337477 wait complete. 2024-09-13T17:41:59.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:41:59.787 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:41:59.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4847a8e640 1 Processor -- start 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4847a8e640 1 -- start start 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4847a8e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f484013af50 0x7f484013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4847a8e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4840138050 con 0x7f484009d690 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4847a8e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484009c410 con 0x7f484013af50 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4845029640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f484013af50 0x7f484013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f484582a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f484009d690 0x7f4840138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43286/0 (socket says 172.21.15.96:43286) 2024-09-13T17:41:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f484582a640 1 -- 172.21.15.96:0/2290972830 learned_addr learned my addr 172.21.15.96:0/2290972830 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:41:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4845029640 1 -- 172.21.15.96:0/2290972830 >> v1:172.21.15.96:6789/0 conn(0x7f484009d690 legacy=0x7f4840138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4845029640 1 -- 172.21.15.96:0/2290972830 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484009cba0 con 0x7f484013af50 2024-09-13T17:41:59.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.898+0000 7f4844828640 1 -- 172.21.15.96:0/2290972830 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007332520 0 0) 0x7f4838001120 con 0x7f484009d690 2024-09-13T17:41:59.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4845029640 1 --2- 172.21.15.96:0/2290972830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484013af50 0x7f484013d340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4834009740 tx=0x7f483401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa6a4742c53ebc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4844828640 1 -- 172.21.15.96:0/2290972830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48340a8840 con 0x7f484013af50 2024-09-13T17:41:59.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4844828640 1 -- 172.21.15.96:0/2290972830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48340a89a0 con 0x7f484013af50 2024-09-13T17:41:59.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4844828640 1 -- 172.21.15.96:0/2290972830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48340a8c60 con 0x7f484013af50 2024-09-13T17:41:59.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- 172.21.15.96:0/2290972830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484013af50 msgr2=0x7f484013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 --2- 172.21.15.96:0/2290972830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484013af50 0x7f484013d340 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4834009740 tx=0x7f483401ec10 comp rx=0 tx=0).stop 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- 172.21.15.96:0/2290972830 shutdown_connections 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 --2- 172.21.15.96:0/2290972830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484013af50 0x7f484013d340 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- 172.21.15.96:0/2290972830 >> 172.21.15.96:0/2290972830 conn(0x7f484006bc60 msgr2=0x7f484006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- 172.21.15.96:0/2290972830 shutdown_connections 2024-09-13T17:41:59.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- 172.21.15.96:0/2290972830 wait complete. 2024-09-13T17:41:59.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 Processor -- start 2024-09-13T17:41:59.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- start start 2024-09-13T17:41:59.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.902+0000 7f4847a8e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400a5c90 con 0x7f484009d690 2024-09-13T17:41:59.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f484582a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f484582a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35042/0 (socket says 172.21.15.96:35042) 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f484582a640 1 -- 172.21.15.96:0/1709748882 learned_addr learned my addr 172.21.15.96:0/1709748882 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f484582a640 1 -- 172.21.15.96:0/1709748882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834009420 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f484582a640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f483800c390 tx=0x7f483800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4838007410 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838007590 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48400a6f20 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f48380047f0 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4840137400 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f4838004950 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 0x7f482c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f4838046f60 con 0x7f484009d690 2024-09-13T17:41:59.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4840138050 con 0x7f484009d690 2024-09-13T17:41:59.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f4845029640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 0x7f482c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:41:59.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.906+0000 7f4845029640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 0x7f482c037910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48340b2000 tx=0x7f48340aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:41:59.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:41:59.910+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4840138050 con 0x7f484009d690 2024-09-13T17:42:00.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.066+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4840138050 con 0x7f484009d690 2024-09-13T17:42:00.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f483e7fc640 1 -- 172.21.15.96:0/1709748882 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v11) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4840138050 con 0x7f484009d690 2024-09-13T17:42:00.068 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 msgr2=0x7f482c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 0x7f482c037910 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f48340b2000 tx=0x7f48340aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 msgr2=0x7f48400a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f483800c390 tx=0x7f483800c850 comp rx=0 tx=0).stop 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 shutdown_connections 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f482c035450 0x7f482c037910 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 --2- 172.21.15.96:0/1709748882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484009d690 0x7f48400a69e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 >> 172.21.15.96:0/1709748882 conn(0x7f484006bc60 msgr2=0x7f484009f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:00.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 shutdown_connections 2024-09-13T17:42:00.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.070+0000 7f4847a8e640 1 -- 172.21.15.96:0/1709748882 wait complete. 2024-09-13T17:42:00.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:42:00.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4df0fd640 1 Processor -- start 2024-09-13T17:42:00.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4df0fd640 1 -- start start 2024-09-13T17:42:00.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4df0fd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe4d809d690 0x7fe4d8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4df0fd640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d8138050 con 0x7fe4d81387d0 2024-09-13T17:42:00.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4df0fd640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d809c410 con 0x7fe4d809d690 2024-09-13T17:42:00.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4dce99640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe4d809d690 0x7fe4d8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4d7fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe4d81387d0 0x7fe4d8134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43290/0 (socket says 172.21.15.96:43290) 2024-09-13T17:42:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4d7fff640 1 -- 172.21.15.96:0/3754012474 learned_addr learned my addr 172.21.15.96:0/3754012474 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4dce99640 1 -- 172.21.15.96:0/3754012474 >> v1:172.21.15.96:6789/0 conn(0x7fe4d81387d0 legacy=0x7fe4d8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4d77fe640 1 -- 172.21.15.96:0/3754012474 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1448342964 0 0) 0x7fe4c8001120 con 0x7fe4d81387d0 2024-09-13T17:42:00.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4dce99640 1 -- 172.21.15.96:0/3754012474 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d809cba0 con 0x7fe4d809d690 2024-09-13T17:42:00.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.194+0000 7fe4dce99640 1 --2- 172.21.15.96:0/3754012474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d8138290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009740 tx=0x7fe4cc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2041f4f71432d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d77fe640 1 -- 172.21.15.96:0/3754012474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc0a8840 con 0x7fe4d809d690 2024-09-13T17:42:00.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d77fe640 1 -- 172.21.15.96:0/3754012474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0a89a0 con 0x7fe4d809d690 2024-09-13T17:42:00.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d77fe640 1 -- 172.21.15.96:0/3754012474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc0a8c60 con 0x7fe4d809d690 2024-09-13T17:42:00.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/3754012474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 msgr2=0x7fe4d8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/3754012474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d8138290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009740 tx=0x7fe4cc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:00.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/3754012474 shutdown_connections 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/3754012474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d8138290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/3754012474 >> 172.21.15.96:0/3754012474 conn(0x7fe4d806bc60 msgr2=0x7fe4d806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/3754012474 shutdown_connections 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/3754012474 wait complete. 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 Processor -- start 2024-09-13T17:42:00.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- start start 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80a61a0 con 0x7fe4d809d690 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4dce99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4dce99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35054/0 (socket says 172.21.15.96:35054) 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4dce99640 1 -- 172.21.15.96:0/4119995726 learned_addr learned my addr 172.21.15.96:0/4119995726 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4dce99640 1 -- 172.21.15.96:0/4119995726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009420 con 0x7fe4d809d690 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4dce99640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01b820 tx=0x7fe4cc01d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc004150 con 0x7fe4d809d690 2024-09-13T17:42:00.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0042b0 con 0x7fe4d809d690 2024-09-13T17:42:00.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4cc0bc720 con 0x7fe4d809d690 2024-09-13T17:42:00.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d80a5ee0 con 0x7fe4d809d690 2024-09-13T17:42:00.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.198+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d809f420 con 0x7fe4d809d690 2024-09-13T17:42:00.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d809bc00 con 0x7fe4d809d690 2024-09-13T17:42:00.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe4cc0bc880 con 0x7fe4d809d690 2024-09-13T17:42:00.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4d57fa640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 0x7fe4c4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7fe4cc0e55f0 con 0x7fe4d809d690 2024-09-13T17:42:00.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4d7fff640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 0x7fe4c4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.202+0000 7fe4d7fff640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 0x7fe4c4037910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8000c10 tx=0x7fe4c8001260 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.206+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe4d809bc00 con 0x7fe4d809d690 2024-09-13T17:42:00.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe4d80a62e0 con 0x7fe4d809d690 2024-09-13T17:42:00.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4d57fa640 1 -- 172.21.15.96:0/4119995726 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v11) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe4d80a62e0 con 0x7fe4d809d690 2024-09-13T17:42:00.353 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:00.353 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:42:00.353 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:00.353 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:42:00.354 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:42:00.354 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:00.354 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:42:00.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 msgr2=0x7fe4c4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 0x7fe4c4037910 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8000c10 tx=0x7fe4c8001260 comp rx=0 tx=0).stop 2024-09-13T17:42:00.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 msgr2=0x7fe4d80a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc01b820 tx=0x7fe4cc01d450 comp rx=0 tx=0).stop 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 shutdown_connections 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe4c4035450 0x7fe4c4037910 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 --2- 172.21.15.96:0/4119995726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4d809d690 0x7fe4d80a6920 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 >> 172.21.15.96:0/4119995726 conn(0x7fe4d806bc60 msgr2=0x7fe4d809e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 shutdown_connections 2024-09-13T17:42:00.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.354+0000 7fe4df0fd640 1 -- 172.21.15.96:0/4119995726 wait complete. 2024-09-13T17:42:00.372 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:42:00.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:42:00.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.482+0000 7f7839789640 1 Processor -- start 2024-09-13T17:42:00.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.482+0000 7f7839789640 1 -- start start 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7834135e80 0x7f783409e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f783409e0d0 con 0x7f7834136260 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783409ef50 con 0x7f7834135e80 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7834135e80 0x7f783409e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7834135e80 0x7f783409e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35062/0 (socket says 172.21.15.96:35062) 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 -- 172.21.15.96:0/1116035109 learned_addr learned my addr 172.21.15.96:0/1116035109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f78327fc640 1 --1- 172.21.15.96:0/1116035109 >> v1:172.21.15.96:6789/0 conn(0x7f7834136260 0x7f783406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43300/0 (socket says 172.21.15.96:43300) 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 -- 172.21.15.96:0/1116035109 >> v1:172.21.15.96:6789/0 conn(0x7f7834136260 legacy=0x7f783406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 -- 172.21.15.96:0/1116035109 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783409bf40 con 0x7f7834135e80 2024-09-13T17:42:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7832ffd640 1 --2- 172.21.15.96:0/1116035109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f783409e890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f781c009e50 tx=0x7f781c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1054b85d12d53449 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7831ffb640 1 -- 172.21.15.96:0/1116035109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781c009200 con 0x7f7834135e80 2024-09-13T17:42:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7831ffb640 1 -- 172.21.15.96:0/1116035109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c0a5790 con 0x7f7834135e80 2024-09-13T17:42:00.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7831ffb640 1 -- 172.21.15.96:0/1116035109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781c0a5a50 con 0x7f7834135e80 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- 172.21.15.96:0/1116035109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 msgr2=0x7f783409e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 --2- 172.21.15.96:0/1116035109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f783409e890 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f781c009e50 tx=0x7f781c01ca30 comp rx=0 tx=0).stop 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- 172.21.15.96:0/1116035109 shutdown_connections 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 --2- 172.21.15.96:0/1116035109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f783409e890 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- 172.21.15.96:0/1116035109 >> 172.21.15.96:0/1116035109 conn(0x7f7834098a60 msgr2=0x7f783409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.486+0000 7f7839789640 1 -- 172.21.15.96:0/1116035109 shutdown_connections 2024-09-13T17:42:00.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- 172.21.15.96:0/1116035109 wait complete. 2024-09-13T17:42:00.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 Processor -- start 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- start start 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78340a3bc0 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7832ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35074/0 (socket says 172.21.15.96:35074) 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7832ffd640 1 -- 172.21.15.96:0/1300738114 learned_addr learned my addr 172.21.15.96:0/1300738114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7832ffd640 1 -- 172.21.15.96:0/1300738114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c009b30 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7832ffd640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f781c01a660 tx=0x7f781c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781c0b87b0 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c0b8910 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f781c0b8bd0 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78340a3ed0 con 0x7f7834135e80 2024-09-13T17:42:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78340a2160 con 0x7f7834135e80 2024-09-13T17:42:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f781c0b8d30 con 0x7f7834135e80 2024-09-13T17:42:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78340a3d00 con 0x7f7834135e80 2024-09-13T17:42:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 0x7f7814037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f781c0e2dc0 con 0x7f7834135e80 2024-09-13T17:42:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f78327fc640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 0x7f7814037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.490+0000 7f78327fc640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 0x7f7814037910 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f782c001600 tx=0x7f782c007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.494+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f781c0c0d80 con 0x7f7834135e80 2024-09-13T17:42:00.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.638+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f78340a2c50 con 0x7f7834135e80 2024-09-13T17:42:00.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.926+0000 7f782b7fe640 1 -- 172.21.15.96:0/1300738114 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v12) v1 ==== 138+0+0 (secure 0 0 0) 0x7f781c0bc030 con 0x7f7834135e80 2024-09-13T17:42:00.925 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-13T17:42:00.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 msgr2=0x7f7814037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 0x7f7814037910 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f782c001600 tx=0x7f782c007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:00.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 msgr2=0x7f78341ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:00.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f781c01a660 tx=0x7f781c0197d0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 shutdown_connections 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7814035450 0x7f7814037910 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 --2- 172.21.15.96:0/1300738114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7834135e80 0x7f78341ca4b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 >> 172.21.15.96:0/1300738114 conn(0x7f7834098a60 msgr2=0x7f783409a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 shutdown_connections 2024-09-13T17:42:00.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:00.930+0000 7f7839789640 1 -- 172.21.15.96:0/1300738114 wait complete. 2024-09-13T17:42:00.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:42:01.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d1e458640 1 Processor -- start 2024-09-13T17:42:01.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d1e458640 1 -- start start 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d1e458640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d18134e70 0x7f0d1809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d1e458640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1809ed60 con 0x7f0d18135250 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d1e458640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1809ddc0 con 0x7f0d18134e70 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d17fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d18134e70 0x7f0d1809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d177fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0d18135250 0x7f0d18135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43316/0 (socket says 172.21.15.96:43316) 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d177fe640 1 -- 172.21.15.96:0/1675324679 learned_addr learned my addr 172.21.15.96:0/1675324679 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:01.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1675324679 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1131250430 0 0) 0x7f0d0c001120 con 0x7f0d18135250 2024-09-13T17:42:01.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1675324679 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d00003530 con 0x7f0d18135250 2024-09-13T17:42:01.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d17fff640 1 -- 172.21.15.96:0/1675324679 >> v1:172.21.15.96:6789/0 conn(0x7f0d18135250 legacy=0x7f0d18135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:01.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.086+0000 7f0d17fff640 1 -- 172.21.15.96:0/1675324679 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d180a68b0 con 0x7f0d18134e70 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 --2- 172.21.15.96:0/1675324679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1809e820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009740 tx=0x7f0d0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f97ef5fe16aba67 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1675324679 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d080ab530 con 0x7f0d18134e70 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1675324679 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d080a7050 con 0x7f0d18134e70 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d16ffd640 1 -- 172.21.15.96:0/1675324679 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d080ab7e0 con 0x7f0d18134e70 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/1675324679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 msgr2=0x7f0d1809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 --2- 172.21.15.96:0/1675324679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1809e820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0d08009740 tx=0x7f0d0801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/1675324679 shutdown_connections 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 --2- 172.21.15.96:0/1675324679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1809e820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/1675324679 >> 172.21.15.96:0/1675324679 conn(0x7f0d18098a60 msgr2=0x7f0d1809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:01.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/1675324679 shutdown_connections 2024-09-13T17:42:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/1675324679 wait complete. 2024-09-13T17:42:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 Processor -- start 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- start start 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d180a3cc0 con 0x7f0d18134e70 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35098/0 (socket says 172.21.15.96:35098) 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 -- 172.21.15.96:0/2890244510 learned_addr learned my addr 172.21.15.96:0/2890244510 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 -- 172.21.15.96:0/2890244510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d08009420 con 0x7f0d18134e70 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d17fff640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d0801a050 tx=0x7f0d080191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d080abdf0 con 0x7f0d18134e70 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d080b4410 con 0x7f0d18134e70 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d18132f70 con 0x7f0d18134e70 2024-09-13T17:42:01.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d080b46b0 con 0x7f0d18134e70 2024-09-13T17:42:01.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.090+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d180a3690 con 0x7f0d18134e70 2024-09-13T17:42:01.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0d080b2040 con 0x7f0d18134e70 2024-09-13T17:42:01.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d14ff9640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 0x7f0d04037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:01.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18071150 con 0x7f0d18134e70 2024-09-13T17:42:01.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2957+0+0 (secure 0 0 0) 0x7f0d080e14a0 con 0x7f0d18134e70 2024-09-13T17:42:01.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d177fe640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 0x7f0d04037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:01.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.094+0000 7f0d177fe640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 0x7f0d04037910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c001630 tx=0x7f0d0c001160 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:01.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.098+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d080b4810 con 0x7f0d18134e70 2024-09-13T17:42:01.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.246+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0d181368b0 con 0x7f0d18134e70 2024-09-13T17:42:01.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.938+0000 7f0d14ff9640 1 -- 172.21.15.96:0/2890244510 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v13) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0d080bbb50 con 0x7f0d18134e70 2024-09-13T17:42:01.938 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-13T17:42:01.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 msgr2=0x7f0d04037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 0x7f0d04037910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c001630 tx=0x7f0d0c001160 comp rx=0 tx=0).stop 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 msgr2=0x7f0d1806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0d0801a050 tx=0x7f0d080191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 shutdown_connections 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0d04035450 0x7f0d04037910 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c001630 tx=0x7f0d0c001160 comp rx=0 tx=0).stop 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 --2- 172.21.15.96:0/2890244510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d18134e70 0x7f0d1806f6d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:01.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 >> 172.21.15.96:0/2890244510 conn(0x7f0d18098a60 msgr2=0x7f0d18098f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:01.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 shutdown_connections 2024-09-13T17:42:01.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:01.946+0000 7f0d1e458640 1 -- 172.21.15.96:0/2890244510 wait complete. 2024-09-13T17:42:01.983 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:42:02.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e4d372640 1 Processor -- start 2024-09-13T17:42:02.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e4d372640 1 -- start start 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e4d372640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e481381b0 0x7f8e480a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e4d372640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4806ca30 con 0x7f8e48138590 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e4d372640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480a02e0 con 0x7f8e481381b0 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e46ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e481381b0 0x7f8e480a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e3ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8e48138590 0x7f8e4806b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43324/0 (socket says 172.21.15.96:43324) 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e3ffff640 1 -- 172.21.15.96:0/2791852337 learned_addr learned my addr 172.21.15.96:0/2791852337 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:02.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e46ffd640 1 -- 172.21.15.96:0/2791852337 >> v1:172.21.15.96:6789/0 conn(0x7f8e48138590 legacy=0x7f8e4806b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e467fc640 1 -- 172.21.15.96:0/2791852337 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194627243 0 0) 0x7f8e4806ca30 con 0x7f8e48138590 2024-09-13T17:42:02.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e46ffd640 1 -- 172.21.15.96:0/2791852337 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480a05a0 con 0x7f8e481381b0 2024-09-13T17:42:02.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.106+0000 7f8e46ffd640 1 --2- 172.21.15.96:0/2791852337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a0a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009740 tx=0x7f8e3001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed4246f17fb1c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e467fc640 1 -- 172.21.15.96:0/2791852337 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e300ab400 con 0x7f8e481381b0 2024-09-13T17:42:02.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e467fc640 1 -- 172.21.15.96:0/2791852337 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e300a7050 con 0x7f8e481381b0 2024-09-13T17:42:02.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e467fc640 1 -- 172.21.15.96:0/2791852337 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e300ab6b0 con 0x7f8e481381b0 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/2791852337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 msgr2=0x7f8e480a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 --2- 172.21.15.96:0/2791852337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a0a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8e30009740 tx=0x7f8e3001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/2791852337 shutdown_connections 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 --2- 172.21.15.96:0/2791852337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a0a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/2791852337 >> 172.21.15.96:0/2791852337 conn(0x7f8e4806ef80 msgr2=0x7f8e480713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/2791852337 shutdown_connections 2024-09-13T17:42:02.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/2791852337 wait complete. 2024-09-13T17:42:02.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 Processor -- start 2024-09-13T17:42:02.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- start start 2024-09-13T17:42:02.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480a6d80 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35112/0 (socket says 172.21.15.96:35112) 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e46ffd640 1 -- 172.21.15.96:0/758116529 learned_addr learned my addr 172.21.15.96:0/758116529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e46ffd640 1 -- 172.21.15.96:0/758116529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30009420 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e46ffd640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e300ae040 tx=0x7f8e3001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e300b7450 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e300b75b0 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e300b7890 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480a8270 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e480a6fb0 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8e300b79f0 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 0x7f8e2c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e480a80c0 con 0x7f8e481381b0 2024-09-13T17:42:02.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.110+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3339+0+0 (secure 0 0 0) 0x7f8e300e1fb0 con 0x7f8e481381b0 2024-09-13T17:42:02.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.114+0000 7f8e3ffff640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 0x7f8e2c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.114+0000 7f8e3ffff640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 0x7f8e2c037910 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e38007fd0 tx=0x7f8e38007940 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.118+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e300b0070 con 0x7f8e481381b0 2024-09-13T17:42:02.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.310+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8e480a53e0 con 0x7f8e481381b0 2024-09-13T17:42:02.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.318+0000 7f8e3f7fe640 1 -- 172.21.15.96:0/758116529 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8e300c78f0 con 0x7f8e481381b0 2024-09-13T17:42:02.319 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-09-13T17:42:02.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 msgr2=0x7f8e2c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 0x7f8e2c037910 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8e38007fd0 tx=0x7f8e38007940 comp rx=0 tx=0).stop 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 msgr2=0x7f8e480a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8e300ae040 tx=0x7f8e3001abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 shutdown_connections 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e2c035450 0x7f8e2c037910 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 --2- 172.21.15.96:0/758116529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e481381b0 0x7f8e480a79f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 >> 172.21.15.96:0/758116529 conn(0x7f8e4806ef80 msgr2=0x7f8e48134ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 shutdown_connections 2024-09-13T17:42:02.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.322+0000 7f8e4d372640 1 -- 172.21.15.96:0/758116529 wait complete. 2024-09-13T17:42:02.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:02.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.470+0000 7f4ba2a48640 1 Processor -- start 2024-09-13T17:42:02.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.470+0000 7f4ba2a48640 1 -- start start 2024-09-13T17:42:02.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b9c134e70 0x7f4b9c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c09ed60 con 0x7f4b9c135250 2024-09-13T17:42:02.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b9c09ddc0 con 0x7f4b9c134e70 2024-09-13T17:42:02.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b9c134e70 0x7f4b9c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b937fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4b9c135250 0x7f4b9c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33586/0 (socket says 172.21.15.96:33586) 2024-09-13T17:42:02.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b937fe640 1 -- 172.21.15.96:0/4213630615 learned_addr learned my addr 172.21.15.96:0/4213630615 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:02.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9bfff640 1 -- 172.21.15.96:0/4213630615 >> v1:172.21.15.96:6789/0 conn(0x7f4b9c135250 legacy=0x7f4b9c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9bfff640 1 -- 172.21.15.96:0/4213630615 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c0a68b0 con 0x7f4b9c134e70 2024-09-13T17:42:02.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9bfff640 1 --2- 172.21.15.96:0/4213630615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c09e820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009740 tx=0x7f4b8c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5debe4415bfee65f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9b7fe640 1 -- 172.21.15.96:0/4213630615 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0ab530 con 0x7f4b9c134e70 2024-09-13T17:42:02.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9b7fe640 1 -- 172.21.15.96:0/4213630615 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0a7050 con 0x7f4b9c134e70 2024-09-13T17:42:02.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4b9b7fe640 1 -- 172.21.15.96:0/4213630615 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0ab7e0 con 0x7f4b9c134e70 2024-09-13T17:42:02.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- 172.21.15.96:0/4213630615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 msgr2=0x7f4b9c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/4213630615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c09e820 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009740 tx=0x7f4b8c01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:02.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- 172.21.15.96:0/4213630615 shutdown_connections 2024-09-13T17:42:02.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/4213630615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c09e820 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- 172.21.15.96:0/4213630615 >> 172.21.15.96:0/4213630615 conn(0x7f4b9c098a60 msgr2=0x7f4b9c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- 172.21.15.96:0/4213630615 shutdown_connections 2024-09-13T17:42:02.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.474+0000 7f4ba2a48640 1 -- 172.21.15.96:0/4213630615 wait complete. 2024-09-13T17:42:02.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 Processor -- start 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 -- start start 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c1201b0 con 0x7f4b9c134e70 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b9bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50378/0 (socket says 172.21.15.96:50378) 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b9bfff640 1 -- 172.21.15.96:0/838951311 learned_addr learned my addr 172.21.15.96:0/838951311 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:02.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b9bfff640 1 -- 172.21.15.96:0/838951311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c009420 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b9bfff640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c01a050 tx=0x7f4b8c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0abea0 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b8c0b4410 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c13a970 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b8c0b46b0 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c11e7e0 con 0x7f4b9c134e70 2024-09-13T17:42:02.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c11fac0 con 0x7f4b9c134e70 2024-09-13T17:42:02.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f4b8c0b2040 con 0x7f4b9c134e70 2024-09-13T17:42:02.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 0x7f4b7c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f4b8c0e1810 con 0x7f4b9c134e70 2024-09-13T17:42:02.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b937fe640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 0x7f4b7c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.478+0000 7f4b937fe640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 0x7f4b7c037910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b84001610 tx=0x7f4b84007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.486+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b9c11fac0 con 0x7f4b9c134e70 2024-09-13T17:42:02.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.646+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b9c11fac0 con 0x7f4b9c134e70 2024-09-13T17:42:02.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4b997fa640 1 -- 172.21.15.96:0/838951311 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10504 (secure 0 0 0) 0x7f4b9c11fac0 con 0x7f4b9c134e70 2024-09-13T17:42:02.648 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:02.649 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:02.312674+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 msgr2=0x7f4b7c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 0x7f4b7c037910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4b84001610 tx=0x7f4b84007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 msgr2=0x7f4b9c11c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c01a050 tx=0x7f4b8c0191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 shutdown_connections 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4b7c035450 0x7f4b7c037910 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 --2- 172.21.15.96:0/838951311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b9c134e70 0x7f4b9c11c930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 >> 172.21.15.96:0/838951311 conn(0x7f4b9c098a60 msgr2=0x7f4b9c0a5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 shutdown_connections 2024-09-13T17:42:02.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.650+0000 7f4ba2a48640 1 -- 172.21.15.96:0/838951311 wait complete. 2024-09-13T17:42:02.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:02.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a9da09640 1 Processor -- start 2024-09-13T17:42:02.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a9da09640 1 -- start start 2024-09-13T17:42:02.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a9da09640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9a9809e820 0x7f9a9813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a9da09640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a980a68b0 con 0x7f9a98134e70 2024-09-13T17:42:02.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a9da09640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9809ddc0 con 0x7f9a9809e820 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a96ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9a98134e70 0x7f9a98135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33588/0 (socket says 172.21.15.96:33588) 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a967fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9a9809e820 0x7f9a9813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a96ffd640 1 -- 172.21.15.96:0/2709293233 learned_addr learned my addr 172.21.15.96:0/2709293233 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.786+0000 7f9a967fc640 1 -- 172.21.15.96:0/2709293233 >> v1:172.21.15.96:6789/0 conn(0x7f9a98134e70 legacy=0x7f9a98135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a95ffb640 1 -- 172.21.15.96:0/2709293233 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171063979 0 0) 0x7f9a980a68b0 con 0x7f9a98134e70 2024-09-13T17:42:02.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a967fc640 1 -- 172.21.15.96:0/2709293233 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98136970 con 0x7f9a9809e820 2024-09-13T17:42:02.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a967fc640 1 --2- 172.21.15.96:0/2709293233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a9813ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009740 tx=0x7f9a8801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9d4fee3b60e6e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a95ffb640 1 -- 172.21.15.96:0/2709293233 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a880ab530 con 0x7f9a9809e820 2024-09-13T17:42:02.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a95ffb640 1 -- 172.21.15.96:0/2709293233 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a880a7050 con 0x7f9a9809e820 2024-09-13T17:42:02.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a95ffb640 1 -- 172.21.15.96:0/2709293233 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a880ab7e0 con 0x7f9a9809e820 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 -- 172.21.15.96:0/2709293233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 msgr2=0x7f9a9813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 --2- 172.21.15.96:0/2709293233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a9813ae50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009740 tx=0x7f9a8801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 -- 172.21.15.96:0/2709293233 shutdown_connections 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 --2- 172.21.15.96:0/2709293233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a9813ae50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 -- 172.21.15.96:0/2709293233 >> 172.21.15.96:0/2709293233 conn(0x7f9a98098a60 msgr2=0x7f9a9809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 -- 172.21.15.96:0/2709293233 shutdown_connections 2024-09-13T17:42:02.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 -- 172.21.15.96:0/2709293233 wait complete. 2024-09-13T17:42:02.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.790+0000 7f9a9da09640 1 Processor -- start 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 -- start start 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9811ca40 con 0x7f9a9809e820 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a96ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50390/0 (socket says 172.21.15.96:50390) 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a96ffd640 1 -- 172.21.15.96:0/1949288946 learned_addr learned my addr 172.21.15.96:0/1949288946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a96ffd640 1 -- 172.21.15.96:0/1949288946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009420 con 0x7f9a9809e820 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a96ffd640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00c420 tx=0x7f9a8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c0074d0 con 0x7f9a9809e820 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c007630 con 0x7f9a9809e820 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a8c004960 con 0x7f9a9809e820 2024-09-13T17:42:02.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9811d6e0 con 0x7f9a9809e820 2024-09-13T17:42:02.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9813a8f0 con 0x7f9a9809e820 2024-09-13T17:42:02.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a980a45e0 con 0x7f9a9809e820 2024-09-13T17:42:02.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f9a8c004ac0 con 0x7f9a9809e820 2024-09-13T17:42:02.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 0x7f9a84037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:02.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9a8c046c10 con 0x7f9a9809e820 2024-09-13T17:42:02.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a967fc640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 0x7f9a84037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:02.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.794+0000 7f9a967fc640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 0x7f9a84037910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ad000 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:02.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.798+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9a8c01ab20 con 0x7f9a9809e820 2024-09-13T17:42:02.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.958+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a9813a430 con 0x7f9a9809e820 2024-09-13T17:42:02.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.958+0000 7f9a7ffff640 1 -- 172.21.15.96:0/1949288946 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10504 (secure 0 0 0) 0x7f9a8c018020 con 0x7f9a9809e820 2024-09-13T17:42:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:02.958 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:02.312674+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 msgr2=0x7f9a84037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 0x7f9a84037910 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9a880ad000 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2024-09-13T17:42:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 msgr2=0x7f9a980a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:02.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00c420 tx=0x7f9a8c00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 shutdown_connections 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9a84035450 0x7f9a84037910 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 --2- 172.21.15.96:0/1949288946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a9809e820 0x7f9a980a26c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 >> 172.21.15.96:0/1949288946 conn(0x7f9a98098a60 msgr2=0x7f9a98133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 shutdown_connections 2024-09-13T17:42:02.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:02.962+0000 7f9a9da09640 1 -- 172.21.15.96:0/1949288946 wait complete. 2024-09-13T17:42:02.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:42:03.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f0803b18640 1 Processor -- start 2024-09-13T17:42:03.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f0803b18640 1 -- start start 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f0803b18640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07fc137490 0x7f07fc139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f0803b18640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07fc071670 con 0x7f07fc134670 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f0803b18640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07fc071360 con 0x7f07fc137490 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07fc137490 0x7f07fc139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f07fc137490 0x7f07fc139880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50392/0 (socket says 172.21.15.96:50392) 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 -- 172.21.15.96:0/2787682884 learned_addr learned my addr 172.21.15.96:0/2787682884 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08018b4640 1 --1- 172.21.15.96:0/2787682884 >> v1:172.21.15.96:6789/0 conn(0x7f07fc134670 0x7f07fc134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33600/0 (socket says 172.21.15.96:33600) 2024-09-13T17:42:03.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 -- 172.21.15.96:0/2787682884 >> v1:172.21.15.96:6789/0 conn(0x7f07fc134670 legacy=0x7f07fc134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 -- 172.21.15.96:0/2787682884 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07fc0a7200 con 0x7f07fc137490 2024-09-13T17:42:03.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08008b2640 1 -- 172.21.15.96:0/2787682884 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204916244 0 0) 0x7f07fc071670 con 0x7f07fc134670 2024-09-13T17:42:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08010b3640 1 --2- 172.21.15.96:0/2787682884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc137490 0x7f07fc139880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097f0 tx=0x7f07ec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c06d4ac40cdd3e87 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08008b2640 1 -- 172.21.15.96:0/2787682884 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07ec0ab530 con 0x7f07fc137490 2024-09-13T17:42:03.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.102+0000 7f08008b2640 1 -- 172.21.15.96:0/2787682884 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07ec0a7050 con 0x7f07fc137490 2024-09-13T17:42:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08008b2640 1 -- 172.21.15.96:0/2787682884 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07ec0ab7e0 con 0x7f07fc137490 2024-09-13T17:42:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2787682884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc137490 msgr2=0x7f07fc139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 --2- 172.21.15.96:0/2787682884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc137490 0x7f07fc139880 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0097f0 tx=0x7f07ec01c450 comp rx=0 tx=0).stop 2024-09-13T17:42:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2787682884 shutdown_connections 2024-09-13T17:42:03.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 --2- 172.21.15.96:0/2787682884 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc137490 0x7f07fc139880 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2787682884 >> 172.21.15.96:0/2787682884 conn(0x7f07fc098a60 msgr2=0x7f07fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2787682884 shutdown_connections 2024-09-13T17:42:03.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2787682884 wait complete. 2024-09-13T17:42:03.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 Processor -- start 2024-09-13T17:42:03.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- start start 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07fc0a2db0 con 0x7f07fc134670 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08018b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08018b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50402/0 (socket says 172.21.15.96:50402) 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08018b4640 1 -- 172.21.15.96:0/2499509760 learned_addr learned my addr 172.21.15.96:0/2499509760 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08018b4640 1 -- 172.21.15.96:0/2499509760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ec0094d0 con 0x7f07fc134670 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f08018b4640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07f800c420 tx=0x7f07f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f80074d0 con 0x7f07fc134670 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f8007630 con 0x7f07fc134670 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07f8004960 con 0x7f07fc134670 2024-09-13T17:42:03.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07fc0a3be0 con 0x7f07fc134670 2024-09-13T17:42:03.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07fc0a0f00 con 0x7f07fc134670 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f07f8004ac0 con 0x7f07fc134670 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07fc0a3a40 con 0x7f07fc134670 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 0x7f07e0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.106+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f07f8046c10 con 0x7f07fc134670 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.110+0000 7f08010b3640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 0x7f07e0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.110+0000 7f08010b3640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 0x7f07e0037910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0ad000 tx=0x7f07ec008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.114+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f07fc0a3a40 con 0x7f07fc134670 2024-09-13T17:42:03.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.298+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f07fc0a3a40 con 0x7f07fc134670 2024-09-13T17:42:03.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.314+0000 7f07f27fc640 1 -- 172.21.15.96:0/2499509760 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f07fc0a3a40 con 0x7f07fc134670 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 msgr2=0x7f07e0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 0x7f07e0037910 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0ad000 tx=0x7f07ec008040 comp rx=0 tx=0).stop 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 msgr2=0x7f07fc0a3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f07f800c420 tx=0x7f07f800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 shutdown_connections 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07e0035450 0x7f07e0037910 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 --2- 172.21.15.96:0/2499509760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07fc134670 0x7f07fc0a3500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 >> 172.21.15.96:0/2499509760 conn(0x7f07fc098a60 msgr2=0x7f07fc06d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 shutdown_connections 2024-09-13T17:42:03.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.318+0000 7f0803b18640 1 -- 172.21.15.96:0/2499509760 wait complete. 2024-09-13T17:42:03.335 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:42:03.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:42:03.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.458+0000 7fd92a7c3640 1 Processor -- start 2024-09-13T17:42:03.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.458+0000 7fd92a7c3640 1 -- start start 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd924134e70 0x7fd92409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92409ed60 con 0x7fd924135250 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92409ddc0 con 0x7fd924134e70 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd923fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd924134e70 0x7fd92409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd9237fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd924135250 0x7fd924135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33606/0 (socket says 172.21.15.96:33606) 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd923fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd924134e70 0x7fd92409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50404/0 (socket says 172.21.15.96:50404) 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd9237fe640 1 -- 172.21.15.96:0/2713409432 learned_addr learned my addr 172.21.15.96:0/2713409432 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd923fff640 1 -- 172.21.15.96:0/2713409432 >> v1:172.21.15.96:6789/0 conn(0x7fd924135250 legacy=0x7fd924135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd923fff640 1 -- 172.21.15.96:0/2713409432 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9240a68b0 con 0x7fd924134e70 2024-09-13T17:42:03.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd923fff640 1 --2- 172.21.15.96:0/2713409432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409e820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd914009740 tx=0x7fd91401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec8909ea7fda9ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd922ffd640 1 -- 172.21.15.96:0/2713409432 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9140ab530 con 0x7fd924134e70 2024-09-13T17:42:03.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd922ffd640 1 -- 172.21.15.96:0/2713409432 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9140a7050 con 0x7fd924134e70 2024-09-13T17:42:03.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd922ffd640 1 -- 172.21.15.96:0/2713409432 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9140ab7e0 con 0x7fd924134e70 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- 172.21.15.96:0/2713409432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 msgr2=0x7fd92409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/2713409432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409e820 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd914009740 tx=0x7fd91401c420 comp rx=0 tx=0).stop 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- 172.21.15.96:0/2713409432 shutdown_connections 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/2713409432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409e820 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- 172.21.15.96:0/2713409432 >> 172.21.15.96:0/2713409432 conn(0x7fd924098a60 msgr2=0x7fd92409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:03.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- 172.21.15.96:0/2713409432 shutdown_connections 2024-09-13T17:42:03.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.462+0000 7fd92a7c3640 1 -- 172.21.15.96:0/2713409432 wait complete. 2024-09-13T17:42:03.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 Processor -- start 2024-09-13T17:42:03.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 -- start start 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240a5e30 con 0x7fd924134e70 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd923fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd923fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50414/0 (socket says 172.21.15.96:50414) 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd923fff640 1 -- 172.21.15.96:0/1818029892 learned_addr learned my addr 172.21.15.96:0/1818029892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd923fff640 1 -- 172.21.15.96:0/1818029892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009420 con 0x7fd924134e70 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd923fff640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd91401a050 tx=0x7fd9140191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9140abea0 con 0x7fd924134e70 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9140b4410 con 0x7fd924134e70 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9240a23f0 con 0x7fd924134e70 2024-09-13T17:42:03.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd9140b46b0 con 0x7fd924134e70 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9240a0890 con 0x7fd924134e70 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd9140b2040 con 0x7fd924134e70 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240a20e0 con 0x7fd924134e70 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 0x7fd910037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fd9140e1810 con 0x7fd924134e70 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd9237fe640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 0x7fd910037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:03.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.466+0000 7fd9237fe640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 0x7fd910037910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd918001610 tx=0x7fd918007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:03.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.474+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd9140bc3b0 con 0x7fd924134e70 2024-09-13T17:42:03.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:03.666+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd9241368b0 con 0x7fd924134e70 2024-09-13T17:42:04.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.326+0000 7fd920ff9640 1 -- 172.21.15.96:0/1818029892 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9140b0020 con 0x7fd924134e70 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 msgr2=0x7fd910037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 0x7fd910037910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd918001610 tx=0x7fd918007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 msgr2=0x7fd92409f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd91401a050 tx=0x7fd9140191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 shutdown_connections 2024-09-13T17:42:04.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd910035450 0x7fd910037910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 --2- 172.21.15.96:0/1818029892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd924134e70 0x7fd92409f280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 >> 172.21.15.96:0/1818029892 conn(0x7fd924098a60 msgr2=0x7fd9240995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 shutdown_connections 2024-09-13T17:42:04.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.330+0000 7fd92a7c3640 1 -- 172.21.15.96:0/1818029892 wait complete. 2024-09-13T17:42:04.352 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:42:04.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:04.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.474+0000 7f4661540640 1 Processor -- start 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.474+0000 7f4661540640 1 -- start start 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4661540640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f465c134e90 0x7f465c071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4661540640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f465c070de0 con 0x7f465c135270 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4661540640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f465c071d60 con 0x7f465c134e90 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f465c134e90 0x7f465c071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f465c134e90 0x7f465c071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50428/0 (socket says 172.21.15.96:50428) 2024-09-13T17:42:04.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 -- 172.21.15.96:0/367774371 learned_addr learned my addr 172.21.15.96:0/367774371 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:04.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465a7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f465c135270 0x7f465c06dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33620/0 (socket says 172.21.15.96:33620) 2024-09-13T17:42:04.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 -- 172.21.15.96:0/367774371 >> v1:172.21.15.96:6789/0 conn(0x7f465c135270 legacy=0x7f465c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 -- 172.21.15.96:0/367774371 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c06f110 con 0x7f465c134e90 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f465affd640 1 --2- 172.21.15.96:0/367774371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c071530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f46500097f0 tx=0x7f465001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c83c8098d8ee44 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4659ffb640 1 -- 172.21.15.96:0/367774371 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46500ab530 con 0x7f465c134e90 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4659ffb640 1 -- 172.21.15.96:0/367774371 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46500a7050 con 0x7f465c134e90 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.478+0000 7f4659ffb640 1 -- 172.21.15.96:0/367774371 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46500ab7e0 con 0x7f465c134e90 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/367774371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 msgr2=0x7f465c071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 --2- 172.21.15.96:0/367774371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c071530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f46500097f0 tx=0x7f465001c450 comp rx=0 tx=0).stop 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/367774371 shutdown_connections 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 --2- 172.21.15.96:0/367774371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c071530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/367774371 >> 172.21.15.96:0/367774371 conn(0x7f465c098a60 msgr2=0x7f465c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/367774371 shutdown_connections 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/367774371 wait complete. 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 Processor -- start 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- start start 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465c0a41e0 con 0x7f465c134e90 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50434/0 (socket says 172.21.15.96:50434) 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465affd640 1 -- 172.21.15.96:0/1204520906 learned_addr learned my addr 172.21.15.96:0/1204520906 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465affd640 1 -- 172.21.15.96:0/1204520906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46500094d0 con 0x7f465c134e90 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465affd640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f465001a080 tx=0x7f465001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46500b6420 con 0x7f465c134e90 2024-09-13T17:42:04.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46500b6580 con 0x7f465c134e90 2024-09-13T17:42:04.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465c0a52a0 con 0x7f465c134e90 2024-09-13T17:42:04.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46500b6840 con 0x7f465c134e90 2024-09-13T17:42:04.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465c13d000 con 0x7f465c134e90 2024-09-13T17:42:04.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f46500b2040 con 0x7f465c134e90 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f465c0a5530 con 0x7f465c134e90 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 0x7f4648037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f46500e1b70 con 0x7f465c134e90 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.482+0000 7f465a7fc640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 0x7f4648037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.486+0000 7f465a7fc640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 0x7f4648037910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f464c001620 tx=0x7f464c007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.490+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f465c0a5530 con 0x7f465c134e90 2024-09-13T17:42:04.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.678+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f465c0a5530 con 0x7f465c134e90 2024-09-13T17:42:04.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.678+0000 7f46437fe640 1 -- 172.21.15.96:0/1204520906 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f465c0a5530 con 0x7f465c134e90 2024-09-13T17:42:04.678 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:04.678 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:04.679 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:04.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 msgr2=0x7f4648037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 0x7f4648037910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f464c001620 tx=0x7f464c007a10 comp rx=0 tx=0).stop 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 msgr2=0x7f465c0a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f465001a080 tx=0x7f465001ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 shutdown_connections 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4648035450 0x7f4648037910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 --2- 172.21.15.96:0/1204520906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f465c134e90 0x7f465c0a1b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 >> 172.21.15.96:0/1204520906 conn(0x7f465c098a60 msgr2=0x7f465c133eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 shutdown_connections 2024-09-13T17:42:04.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.682+0000 7f4661540640 1 -- 172.21.15.96:0/1204520906 wait complete. 2024-09-13T17:42:04.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:04.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.802+0000 7fcef4596640 1 Processor -- start 2024-09-13T17:42:04.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.802+0000 7fcef4596640 1 -- start start 2024-09-13T17:42:04.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef4596640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fceec09d690 0x7fceec138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef4596640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fceec138050 con 0x7fceec1387d0 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef4596640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fceec09c410 con 0x7fceec09d690 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef2332640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fceec09d690 0x7fceec138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef1b31640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fceec1387d0 0x7fceec134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33630/0 (socket says 172.21.15.96:33630) 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef2332640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fceec09d690 0x7fceec138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50436/0 (socket says 172.21.15.96:50436) 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef1b31640 1 -- 172.21.15.96:0/1089566335 learned_addr learned my addr 172.21.15.96:0/1089566335 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef2332640 1 -- 172.21.15.96:0/1089566335 >> v1:172.21.15.96:6789/0 conn(0x7fceec1387d0 legacy=0x7fceec134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef1330640 1 -- 172.21.15.96:0/1089566335 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312971285 0 0) 0x7fcedc001120 con 0x7fceec1387d0 2024-09-13T17:42:04.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.806+0000 7fcef2332640 1 -- 172.21.15.96:0/1089566335 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec09cba0 con 0x7fceec09d690 2024-09-13T17:42:04.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef2332640 1 --2- 172.21.15.96:0/1089566335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec138290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009d90 tx=0x7fcee801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488e9ffa7518beef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef1330640 1 -- 172.21.15.96:0/1089566335 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcee80a8e10 con 0x7fceec09d690 2024-09-13T17:42:04.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef1330640 1 -- 172.21.15.96:0/1089566335 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80b0440 con 0x7fceec09d690 2024-09-13T17:42:04.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef1330640 1 -- 172.21.15.96:0/1089566335 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcee80b0700 con 0x7fceec09d690 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- 172.21.15.96:0/1089566335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 msgr2=0x7fceec138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 --2- 172.21.15.96:0/1089566335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec138290 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009d90 tx=0x7fcee801f150 comp rx=0 tx=0).stop 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- 172.21.15.96:0/1089566335 shutdown_connections 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 --2- 172.21.15.96:0/1089566335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec138290 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- 172.21.15.96:0/1089566335 >> 172.21.15.96:0/1089566335 conn(0x7fceec06bc60 msgr2=0x7fceec06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- 172.21.15.96:0/1089566335 shutdown_connections 2024-09-13T17:42:04.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- 172.21.15.96:0/1089566335 wait complete. 2024-09-13T17:42:04.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 Processor -- start 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- start start 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef4596640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec0a61a0 con 0x7fceec09d690 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef2332640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef2332640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50448/0 (socket says 172.21.15.96:50448) 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef2332640 1 -- 172.21.15.96:0/2034911985 learned_addr learned my addr 172.21.15.96:0/2034911985 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.810+0000 7fcef2332640 1 -- 172.21.15.96:0/2034911985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee8009a70 con 0x7fceec09d690 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef2332640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcee801bd60 tx=0x7fcee801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcee80ba540 con 0x7fceec09d690 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee80ba6a0 con 0x7fceec09d690 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec0a5ee0 con 0x7fceec09d690 2024-09-13T17:42:04.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcee80ba9c0 con 0x7fceec09d690 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceec09f420 con 0x7fceec09d690 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcee80b7050 con 0x7fceec09d690 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceec09bc00 con 0x7fceec09d690 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 0x7fced0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fcee80e5db0 con 0x7fceec09d690 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef1b31640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 0x7fced0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:04.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.814+0000 7fcef1b31640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 0x7fced0037910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000c10 tx=0x7fcedc001260 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:04.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.818+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fceec09bc00 con 0x7fceec09d690 2024-09-13T17:42:04.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.982+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fceec0a62e0 con 0x7fceec09d690 2024-09-13T17:42:04.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.982+0000 7fcee2ffd640 1 -- 172.21.15.96:0/2034911985 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fceec0a62e0 con 0x7fceec09d690 2024-09-13T17:42:04.982 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:04.982 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:04.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 msgr2=0x7fced0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 0x7fced0037910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000c10 tx=0x7fcedc001260 comp rx=0 tx=0).stop 2024-09-13T17:42:04.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 msgr2=0x7fceec0a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:04.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcee801bd60 tx=0x7fcee801bef0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 shutdown_connections 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fced0035450 0x7fced0037910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 --2- 172.21.15.96:0/2034911985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fceec09d690 0x7fceec0a6920 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 >> 172.21.15.96:0/2034911985 conn(0x7fceec06bc60 msgr2=0x7fceec09e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 shutdown_connections 2024-09-13T17:42:04.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:04.986+0000 7fcef4596640 1 -- 172.21.15.96:0/2034911985 wait complete. 2024-09-13T17:42:05.000 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:42:05.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:05.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adf4b3640 1 Processor -- start 2024-09-13T17:42:05.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adf4b3640 1 -- start start 2024-09-13T17:42:05.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adf4b3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9ad809e820 0x7f9ad813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adf4b3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80a68b0 con 0x7f9ad8134e70 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adf4b3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad809ddc0 con 0x7f9ad809e820 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adca4e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9ad809e820 0x7f9ad813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adca4e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9ad809e820 0x7f9ad813ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50456/0 (socket says 172.21.15.96:50456) 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9add24f640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9ad8134e70 0x7f9ad8135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33640/0 (socket says 172.21.15.96:33640) 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.106+0000 7f9adca4e640 1 -- 172.21.15.96:0/3384985613 learned_addr learned my addr 172.21.15.96:0/3384985613 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adca4e640 1 -- 172.21.15.96:0/3384985613 >> v1:172.21.15.96:6789/0 conn(0x7f9ad8134e70 legacy=0x7f9ad8135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adca4e640 1 -- 172.21.15.96:0/3384985613 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad8136970 con 0x7f9ad809e820 2024-09-13T17:42:05.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9acffff640 1 -- 172.21.15.96:0/3384985613 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1265278754 0 0) 0x7f9ad80a68b0 con 0x7f9ad8134e70 2024-09-13T17:42:05.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adca4e640 1 --2- 172.21.15.96:0/3384985613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad813ae50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097f0 tx=0x7f9ad401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada6787e61cd219 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9acffff640 1 -- 172.21.15.96:0/3384985613 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad40ac540 con 0x7f9ad809e820 2024-09-13T17:42:05.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9acffff640 1 -- 172.21.15.96:0/3384985613 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad40ac6a0 con 0x7f9ad809e820 2024-09-13T17:42:05.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9acffff640 1 -- 172.21.15.96:0/3384985613 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ad40ac960 con 0x7f9ad809e820 2024-09-13T17:42:05.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- 172.21.15.96:0/3384985613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 msgr2=0x7f9ad813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/3384985613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad813ae50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097f0 tx=0x7f9ad401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- 172.21.15.96:0/3384985613 shutdown_connections 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/3384985613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad813ae50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- 172.21.15.96:0/3384985613 >> 172.21.15.96:0/3384985613 conn(0x7f9ad8098a60 msgr2=0x7f9ad809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- 172.21.15.96:0/3384985613 shutdown_connections 2024-09-13T17:42:05.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- 172.21.15.96:0/3384985613 wait complete. 2024-09-13T17:42:05.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 Processor -- start 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.110+0000 7f9adf4b3640 1 -- start start 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adf4b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adf4b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8071570 con 0x7f9ad809e820 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9add24f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9add24f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50458/0 (socket says 172.21.15.96:50458) 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9add24f640 1 -- 172.21.15.96:0/1698163094 learned_addr learned my addr 172.21.15.96:0/1698163094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9add24f640 1 -- 172.21.15.96:0/1698163094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad40094d0 con 0x7f9ad809e820 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9add24f640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800c420 tx=0x7f9ac800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ac80074d0 con 0x7f9ad809e820 2024-09-13T17:42:05.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac8007630 con 0x7f9ad809e820 2024-09-13T17:42:05.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8071310 con 0x7f9ad809e820 2024-09-13T17:42:05.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ac8004960 con 0x7f9ad809e820 2024-09-13T17:42:05.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80a42e0 con 0x7f9ad809e820 2024-09-13T17:42:05.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f9ac8004ac0 con 0x7f9ad809e820 2024-09-13T17:42:05.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad8132f50 con 0x7f9ad809e820 2024-09-13T17:42:05.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 0x7f9abc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9ac8046c30 con 0x7f9ad809e820 2024-09-13T17:42:05.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adca4e640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 0x7f9abc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.114+0000 7f9adca4e640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 0x7f9abc037910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae000 tx=0x7f9ad4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.118+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ac804d080 con 0x7f9ad809e820 2024-09-13T17:42:05.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.258+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9ad80716c0 con 0x7f9abc035450 2024-09-13T17:42:05.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9acdffb640 1 -- 172.21.15.96:0/1698163094 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+234802 (secure 0 0 0) 0x7f9ad40ac810 con 0x7f9abc035450 2024-09-13T17:42:05.270 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:05.272 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 msgr2=0x7f9abc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 0x7f9abc037910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae000 tx=0x7f9ad4008040 comp rx=0 tx=0).stop 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 msgr2=0x7f9ad81cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800c420 tx=0x7f9ac800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 shutdown_connections 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9abc035450 0x7f9abc037910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 --2- 172.21.15.96:0/1698163094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ad809e820 0x7f9ad81cfa20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 >> 172.21.15.96:0/1698163094 conn(0x7f9ad8098a60 msgr2=0x7f9ad806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 shutdown_connections 2024-09-13T17:42:05.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.270+0000 7f9adf4b3640 1 -- 172.21.15.96:0/1698163094 wait complete. 2024-09-13T17:42:05.286 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T17:42:03.795677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":82696,"kb_used_data":1448,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032824,"statfs":{"total":289910292480,"available":289825611776,"internally_reserved":0,"allocated":1482752,"data_stored":1253714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000909"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":137,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-09-13T17:41:53.903032+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:41:53.903032+0000","last_peered":"2024-09-13T17:41:53.903032+0000","last_clean":"2024-09-13T17:41:53.903032+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:41:53.903032+0000","last_undegraded":"2024-09-13T17:41:53.903032+0000","last_fullsized":"2024-09-13T17:41:53.903032+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":24326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:05.290 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1ed302640 1 Processor -- start 2024-09-13T17:42:05.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1ed302640 1 -- start start 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1ed302640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa1e809d690 0x7fa1e8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1ed302640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e8138050 con 0x7fa1e81387d0 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1ed302640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e809c410 con 0x7fa1e809d690 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e67fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa1e81387d0 0x7fa1e8134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33656/0 (socket says 172.21.15.96:33656) 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e67fc640 1 -- 172.21.15.96:0/1534045025 learned_addr learned my addr 172.21.15.96:0/1534045025 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e6ffd640 1 --2- 172.21.15.96:0/1534045025 >> v2:172.21.15.96:3300/0 conn(0x7fa1e809d690 0x7fa1e8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e5ffb640 1 -- 172.21.15.96:0/1534045025 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30881939 0 0) 0x7fa1d8001120 con 0x7fa1e81387d0 2024-09-13T17:42:05.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e5ffb640 1 -- 172.21.15.96:0/1534045025 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d4003530 con 0x7fa1e81387d0 2024-09-13T17:42:05.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e6ffd640 1 -- 172.21.15.96:0/1534045025 >> v1:172.21.15.96:6789/0 conn(0x7fa1e81387d0 legacy=0x7fa1e8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.438+0000 7fa1e6ffd640 1 -- 172.21.15.96:0/1534045025 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e809cba0 con 0x7fa1e809d690 2024-09-13T17:42:05.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.442+0000 7fa1e6ffd640 1 --2- 172.21.15.96:0/1534045025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e8138290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009740 tx=0x7fa1d001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62657242daccbbe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.442+0000 7fa1e5ffb640 1 -- 172.21.15.96:0/1534045025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d00a8840 con 0x7fa1e809d690 2024-09-13T17:42:05.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.442+0000 7fa1e5ffb640 1 -- 172.21.15.96:0/1534045025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d00a89a0 con 0x7fa1e809d690 2024-09-13T17:42:05.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.442+0000 7fa1e5ffb640 1 -- 172.21.15.96:0/1534045025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d00a8c60 con 0x7fa1e809d690 2024-09-13T17:42:05.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.442+0000 7fa1ed302640 1 -- 172.21.15.96:0/1534045025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 msgr2=0x7fa1e8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 --2- 172.21.15.96:0/1534045025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e8138290 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009740 tx=0x7fa1d001ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:05.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/1534045025 shutdown_connections 2024-09-13T17:42:05.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 --2- 172.21.15.96:0/1534045025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e8138290 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/1534045025 >> 172.21.15.96:0/1534045025 conn(0x7fa1e806bc60 msgr2=0x7fa1e806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/1534045025 shutdown_connections 2024-09-13T17:42:05.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/1534045025 wait complete. 2024-09-13T17:42:05.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 Processor -- start 2024-09-13T17:42:05.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- start start 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e80a63a0 con 0x7fa1e809d690 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50490/0 (socket says 172.21.15.96:50490) 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e6ffd640 1 -- 172.21.15.96:0/236216406 learned_addr learned my addr 172.21.15.96:0/236216406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:05.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e6ffd640 1 -- 172.21.15.96:0/236216406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d0009420 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e6ffd640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa1d001c840 tx=0x7fa1d001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d00b0780 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d00b08e0 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e80a4cd0 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1d00b0b80 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e80a5c40 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fa1d00b7040 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 0x7fa1c8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fa1d00e5300 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e80a4460 con 0x7fa1e809d690 2024-09-13T17:42:05.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e67fc640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 0x7fa1c8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.446+0000 7fa1e67fc640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 0x7fa1c8037910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005890 tx=0x7fa1d8001190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.450+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1d00b5030 con 0x7fa1e809d690 2024-09-13T17:42:05.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.594+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1e809c100 con 0x7fa1c8035450 2024-09-13T17:42:05.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.610+0000 7fa1df7fe640 1 -- 172.21.15.96:0/236216406 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+234802 (secure 0 0 0) 0x7fa1d8001c00 con 0x7fa1c8035450 2024-09-13T17:42:05.612 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:05.613 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:05.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 msgr2=0x7fa1c8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 0x7fa1c8037910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8005890 tx=0x7fa1d8001190 comp rx=0 tx=0).stop 2024-09-13T17:42:05.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 msgr2=0x7fa1e80a6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa1d001c840 tx=0x7fa1d001b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 shutdown_connections 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa1c8035450 0x7fa1c8037910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 --2- 172.21.15.96:0/236216406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1e809d690 0x7fa1e80a6970 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 >> 172.21.15.96:0/236216406 conn(0x7fa1e806bc60 msgr2=0x7fa1e80994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 shutdown_connections 2024-09-13T17:42:05.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.614+0000 7fa1ed302640 1 -- 172.21.15.96:0/236216406 wait complete. 2024-09-13T17:42:05.628 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-13T17:42:03.795677+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":82696,"kb_used_data":1448,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032824,"statfs":{"total":289910292480,"available":289825611776,"internally_reserved":0,"allocated":1482752,"data_stored":1253714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000909"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":137,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-09-13T17:41:53.903032+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:41:53.903032+0000","last_peered":"2024-09-13T17:41:53.903032+0000","last_clean":"2024-09-13T17:41:53.903032+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:41:53.903032+0000","last_undegraded":"2024-09-13T17:41:53.903032+0000","last_fullsized":"2024-09-13T17:41:53.903032+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":24326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:05.631 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e99c2640 1 Processor -- start 2024-09-13T17:42:05.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e99c2640 1 -- start start 2024-09-13T17:42:05.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e99c2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34e409e820 0x7f34e413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e99c2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34e40a68b0 con 0x7f34e4134e70 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e99c2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34e409ddc0 con 0x7f34e409e820 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e27fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34e409e820 0x7f34e413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e27fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34e409e820 0x7f34e413ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50496/0 (socket says 172.21.15.96:50496) 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e27fc640 1 -- 172.21.15.96:0/4236613121 learned_addr learned my addr 172.21.15.96:0/4236613121 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e27fc640 1 -- 172.21.15.96:0/4236613121 >> v1:172.21.15.96:6789/0 conn(0x7f34e4134e70 legacy=0x7f34e4135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.786+0000 7f34e27fc640 1 -- 172.21.15.96:0/4236613121 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4136970 con 0x7f34e409e820 2024-09-13T17:42:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e27fc640 1 --2- 172.21.15.96:0/4236613121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e413ae50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097f0 tx=0x7f34cc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b328767b7e5b99d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e1ffb640 1 -- 172.21.15.96:0/4236613121 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34cc0ac540 con 0x7f34e409e820 2024-09-13T17:42:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e1ffb640 1 -- 172.21.15.96:0/4236613121 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc0ac6a0 con 0x7f34e409e820 2024-09-13T17:42:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e1ffb640 1 -- 172.21.15.96:0/4236613121 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34cc0ac960 con 0x7f34e409e820 2024-09-13T17:42:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- 172.21.15.96:0/4236613121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 msgr2=0x7f34e413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 --2- 172.21.15.96:0/4236613121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e413ae50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0097f0 tx=0x7f34cc01c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- 172.21.15.96:0/4236613121 shutdown_connections 2024-09-13T17:42:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 --2- 172.21.15.96:0/4236613121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e413ae50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- 172.21.15.96:0/4236613121 >> 172.21.15.96:0/4236613121 conn(0x7f34e4098a60 msgr2=0x7f34e409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- 172.21.15.96:0/4236613121 shutdown_connections 2024-09-13T17:42:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- 172.21.15.96:0/4236613121 wait complete. 2024-09-13T17:42:05.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 Processor -- start 2024-09-13T17:42:05.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.790+0000 7f34e99c2640 1 -- start start 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e99c2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e99c2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e40a44c0 con 0x7f34e409e820 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50510/0 (socket says 172.21.15.96:50510) 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e2ffd640 1 -- 172.21.15.96:0/2093432482 learned_addr learned my addr 172.21.15.96:0/2093432482 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e2ffd640 1 -- 172.21.15.96:0/2093432482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0094d0 con 0x7f34e409e820 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e2ffd640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c7f0 tx=0x7f34dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc006c90 con 0x7f34e409e820 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc006e10 con 0x7f34e409e820 2024-09-13T17:42:05.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40a1960 con 0x7f34e409e820 2024-09-13T17:42:05.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34dc00a960 con 0x7f34e409e820 2024-09-13T17:42:05.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e40a41f0 con 0x7f34e409e820 2024-09-13T17:42:05.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e40008e0 con 0x7f34e409e820 2024-09-13T17:42:05.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f34dc00aac0 con 0x7f34e409e820 2024-09-13T17:42:05.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 0x7f34c4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:05.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f34dc047100 con 0x7f34e409e820 2024-09-13T17:42:05.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e27fc640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 0x7f34c4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:05.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.794+0000 7f34e27fc640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 0x7f34c4037910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0ae000 tx=0x7f34cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:05.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.798+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f34dc019310 con 0x7f34e409e820 2024-09-13T17:42:05.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.954+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34e40a66b0 con 0x7f34c4035450 2024-09-13T17:42:05.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.958+0000 7f34dbfff640 1 -- 172.21.15.96:0/2093432482 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+234802 (secure 0 0 0) 0x7f34e40a66b0 con 0x7f34c4035450 2024-09-13T17:42:05.961 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:05.963 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 msgr2=0x7f34c4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 0x7f34c4037910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0ae000 tx=0x7f34cc008040 comp rx=0 tx=0).stop 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 msgr2=0x7f34e41cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f34dc00c7f0 tx=0x7f34dc00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 shutdown_connections 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34c4035450 0x7f34c4037910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 --2- 172.21.15.96:0/2093432482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34e409e820 0x7f34e41cdae0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:05.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 >> 172.21.15.96:0/2093432482 conn(0x7f34e4098a60 msgr2=0x7f34e4098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:05.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 shutdown_connections 2024-09-13T17:42:05.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:05.962+0000 7f34e99c2640 1 -- 172.21.15.96:0/2093432482 wait complete. 2024-09-13T17:42:05.979 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T17:42:05.796138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":82696,"kb_used_data":1448,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032824,"statfs":{"total":289910292480,"available":289825611776,"internally_reserved":0,"allocated":1482752,"data_stored":1253714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.872784"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":137,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-09-13T17:41:53.903032+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:41:53.903032+0000","last_peered":"2024-09-13T17:41:53.903032+0000","last_clean":"2024-09-13T17:41:53.903032+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:41:53.903032+0000","last_undegraded":"2024-09-13T17:41:53.903032+0000","last_fullsized":"2024-09-13T17:41:53.903032+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":24326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:05.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.984 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.985 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2024-09-13T17:42:05.986 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.987 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.988 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.989 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2024-09-13T17:42:05.990 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.991 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2024-09-13T17:42:05.992 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2024-09-13T17:42:05.993 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.994 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2024-09-13T17:42:05.994 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2024-09-13T17:42:05.995 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.996 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2024-09-13T17:42:05.997 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2024-09-13T17:42:05.998 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:05.999 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2024-09-13T17:42:06.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.000 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2024-09-13T17:42:06.000 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2024-09-13T17:42:06.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2024-09-13T17:42:06.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2024-09-13T17:42:06.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:01.930280+0000', 'last_change': '2024-09-13T17:42:01.930280+0000', 'last_active': '2024-09-13T17:42:01.930280+0000', 'last_peered': '2024-09-13T17:42:01.930280+0000', 'last_clean': '2024-09-13T17:42:01.930280+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:01.930280+0000', 'last_undegraded': '2024-09-13T17:42:01.930280+0000', 'last_fullsized': '2024-09-13T17:42:01.930280+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:01.930280+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2024-09-13T17:42:06.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-13T17:42:00.918750+0000', 'last_change': '2024-09-13T17:42:00.918750+0000', 'last_active': '2024-09-13T17:42:00.918750+0000', 'last_peered': '2024-09-13T17:42:00.918750+0000', 'last_clean': '2024-09-13T17:42:00.918750+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:00.918750+0000', 'last_undegraded': '2024-09-13T17:42:00.918750+0000', 'last_fullsized': '2024-09-13T17:42:00.918750+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:00.918750+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-13T17:42:06.003 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T17:42:06.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34fbb75640 1 Processor -- start 2024-09-13T17:42:06.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34fbb75640 1 -- start start 2024-09-13T17:42:06.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34fbb75640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34f4134e70 0x7f34f409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34fbb75640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f409ed60 con 0x7f34f4135250 2024-09-13T17:42:06.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34fbb75640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f409ddc0 con 0x7f34f4134e70 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9911640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34f4134e70 0x7f34f409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9110640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f34f4135250 0x7f34f4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33670/0 (socket says 172.21.15.96:33670) 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9911640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f34f4134e70 0x7f34f409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50512/0 (socket says 172.21.15.96:50512) 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9110640 1 -- 172.21.15.96:0/567688722 learned_addr learned my addr 172.21.15.96:0/567688722 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9911640 1 -- 172.21.15.96:0/567688722 >> v1:172.21.15.96:6789/0 conn(0x7f34f4135250 legacy=0x7f34f4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f9911640 1 -- 172.21.15.96:0/567688722 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40a68b0 con 0x7f34f4134e70 2024-09-13T17:42:06.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.130+0000 7f34f890f640 1 -- 172.21.15.96:0/567688722 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418121234 0 0) 0x7f34e4001120 con 0x7f34f4135250 2024-09-13T17:42:06.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 --2- 172.21.15.96:0/567688722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f409e820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009740 tx=0x7f34ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9a7c7b665b8f3e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f890f640 1 -- 172.21.15.96:0/567688722 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec0ab530 con 0x7f34f4134e70 2024-09-13T17:42:06.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f890f640 1 -- 172.21.15.96:0/567688722 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec0a7050 con 0x7f34f4134e70 2024-09-13T17:42:06.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f890f640 1 -- 172.21.15.96:0/567688722 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec0ab7e0 con 0x7f34f4134e70 2024-09-13T17:42:06.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/567688722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 msgr2=0x7f34f409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 --2- 172.21.15.96:0/567688722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f409e820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009740 tx=0x7f34ec01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/567688722 shutdown_connections 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 --2- 172.21.15.96:0/567688722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f409e820 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/567688722 >> 172.21.15.96:0/567688722 conn(0x7f34f4098a60 msgr2=0x7f34f409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/567688722 shutdown_connections 2024-09-13T17:42:06.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/567688722 wait complete. 2024-09-13T17:42:06.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 Processor -- start 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- start start 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f4132e00 con 0x7f34f4134e70 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50518/0 (socket says 172.21.15.96:50518) 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 -- 172.21.15.96:0/391026719 learned_addr learned my addr 172.21.15.96:0/391026719 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 -- 172.21.15.96:0/391026719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec009420 con 0x7f34f4134e70 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34f9911640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34ec019030 tx=0x7f34ec01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec0b7450 con 0x7f34f4134e70 2024-09-13T17:42:06.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec0b75b0 con 0x7f34f4134e70 2024-09-13T17:42:06.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f412fa60 con 0x7f34f4134e70 2024-09-13T17:42:06.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.134+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34ec0b7890 con 0x7f34f4134e70 2024-09-13T17:42:06.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f40a4590 con 0x7f34f4134e70 2024-09-13T17:42:06.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f34ec0b79f0 con 0x7f34f4134e70 2024-09-13T17:42:06.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f41368b0 con 0x7f34f4134e70 2024-09-13T17:42:06.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34ea7fc640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 0x7f34d8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f34ec0e2000 con 0x7f34f4134e70 2024-09-13T17:42:06.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34f9110640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 0x7f34d8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.138+0000 7f34f9110640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 0x7f34d8037910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34e4007fd0 tx=0x7f34e4007bc0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.142+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f34ec0c74e0 con 0x7f34f4134e70 2024-09-13T17:42:06.275 INFO:tasks.ceph.mds.a.smithi096.stderr:2024-09-13T17:42:06.274+0000 7f2853a7f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:42:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.346+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f34f406a1b0 con 0x7f34f4134e70 2024-09-13T17:42:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.346+0000 7f34ea7fc640 1 -- 172.21.15.96:0/391026719 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1007 (secure 0 0 0) 0x7f34ec0b0080 con 0x7f34f4134e70 2024-09-13T17:42:06.348 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:06.348 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":25,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":15,"num_osds":3,"num_up_osds":3,"osd_up_since":1726249310,"num_in_osds":3,"osd_in_since":1726249301,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":84680704,"bytes_avail":289825611776,"bytes_total":289910292480,"unknown_pgs_ratio":0.98969072103500366},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4145}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-13T17:41:41.091100+0000","services":{}},"progress_events":{}} 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 msgr2=0x7f34d8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 0x7f34d8037910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f34e4007fd0 tx=0x7f34e4007bc0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 msgr2=0x7f34f40a3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f34ec019030 tx=0x7f34ec01ac60 comp rx=0 tx=0).stop 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 shutdown_connections 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f34d8035450 0x7f34d8037910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 --2- 172.21.15.96:0/391026719 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34f4134e70 0x7f34f40a3b00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 >> 172.21.15.96:0/391026719 conn(0x7f34f4098a60 msgr2=0x7f34f4133010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 shutdown_connections 2024-09-13T17:42:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.350+0000 7f34fbb75640 1 -- 172.21.15.96:0/391026719 wait complete. 2024-09-13T17:42:06.364 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-09-13T17:42:06.365 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:06.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b49d5e640 1 Processor -- start 2024-09-13T17:42:06.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b49d5e640 1 -- start start 2024-09-13T17:42:06.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b49d5e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6b44134e70 0x7f6b4409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b49d5e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b4409ed60 con 0x7f6b44135250 2024-09-13T17:42:06.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b49d5e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b4409ddc0 con 0x7f6b44134e70 2024-09-13T17:42:06.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b42ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6b44135250 0x7f6b44135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33686/0 (socket says 172.21.15.96:33686) 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b42ffd640 1 -- 172.21.15.96:0/859996704 learned_addr learned my addr 172.21.15.96:0/859996704 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b437fe640 1 --2- 172.21.15.96:0/859996704 >> v2:172.21.15.96:3300/0 conn(0x7f6b44134e70 0x7f6b4409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b427fc640 1 -- 172.21.15.96:0/859996704 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418113418 0 0) 0x7f6b2c001120 con 0x7f6b44135250 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b427fc640 1 -- 172.21.15.96:0/859996704 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b30003530 con 0x7f6b44135250 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b437fe640 1 -- 172.21.15.96:0/859996704 >> v1:172.21.15.96:6789/0 conn(0x7f6b44135250 legacy=0x7f6b44135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.478+0000 7f6b437fe640 1 -- 172.21.15.96:0/859996704 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b440a68b0 con 0x7f6b44134e70 2024-09-13T17:42:06.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b437fe640 1 --2- 172.21.15.96:0/859996704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4409e820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009d90 tx=0x7f6b3c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95734d7cb3fdc1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b427fc640 1 -- 172.21.15.96:0/859996704 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c0a57b0 con 0x7f6b44134e70 2024-09-13T17:42:06.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b427fc640 1 -- 172.21.15.96:0/859996704 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0a5910 con 0x7f6b44134e70 2024-09-13T17:42:06.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b427fc640 1 -- 172.21.15.96:0/859996704 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c0a5bd0 con 0x7f6b44134e70 2024-09-13T17:42:06.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- 172.21.15.96:0/859996704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 msgr2=0x7f6b4409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/859996704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4409e820 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009d90 tx=0x7f6b3c01c970 comp rx=0 tx=0).stop 2024-09-13T17:42:06.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- 172.21.15.96:0/859996704 shutdown_connections 2024-09-13T17:42:06.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/859996704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4409e820 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- 172.21.15.96:0/859996704 >> 172.21.15.96:0/859996704 conn(0x7f6b44098a60 msgr2=0x7f6b4409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- 172.21.15.96:0/859996704 shutdown_connections 2024-09-13T17:42:06.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- 172.21.15.96:0/859996704 wait complete. 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 Processor -- start 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- start start 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.482+0000 7f6b49d5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b440a4f90 con 0x7f6b44134e70 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50540/0 (socket says 172.21.15.96:50540) 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b437fe640 1 -- 172.21.15.96:0/121404550 learned_addr learned my addr 172.21.15.96:0/121404550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b437fe640 1 -- 172.21.15.96:0/121404550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b3c009a70 con 0x7f6b44134e70 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b437fe640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c01a5a0 tx=0x7f6b3c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c0b9580 con 0x7f6b44134e70 2024-09-13T17:42:06.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3c0b96e0 con 0x7f6b44134e70 2024-09-13T17:42:06.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b44070800 con 0x7f6b44134e70 2024-09-13T17:42:06.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6b3c0b99c0 con 0x7f6b44134e70 2024-09-13T17:42:06.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b440711e0 con 0x7f6b44134e70 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f6b3c0b9b20 con 0x7f6b44134e70 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 0x7f6b240378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f6b3c0e2d20 con 0x7f6b44134e70 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b441368b0 con 0x7f6b44134e70 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 0x7f6b240378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:06.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.486+0000 7f6b42ffd640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 0x7f6b240378c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c001630 tx=0x7f6b2c001160 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:06.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.490+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6b3c0bd070 con 0x7f6b44134e70 2024-09-13T17:42:06.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.630+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6b44136580 con 0x7f6b24035400 2024-09-13T17:42:06.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.646+0000 7f6b3bfff640 1 -- 172.21.15.96:0/121404550 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+234802 (secure 0 0 0) 0x7f6b2c0058c0 con 0x7f6b24035400 2024-09-13T17:42:06.647 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:06.650 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 msgr2=0x7f6b240378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 0x7f6b240378c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c001630 tx=0x7f6b2c001160 comp rx=0 tx=0).stop 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 msgr2=0x7f6b4406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c01a5a0 tx=0x7f6b3c019710 comp rx=0 tx=0).stop 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 shutdown_connections 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6b24035400 0x7f6b240378c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 --2- 172.21.15.96:0/121404550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b44134e70 0x7f6b4406f6d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 >> 172.21.15.96:0/121404550 conn(0x7f6b44098a60 msgr2=0x7f6b44098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:06.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 shutdown_connections 2024-09-13T17:42:06.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:06.650+0000 7f6b49d5e640 1 -- 172.21.15.96:0/121404550 wait complete. 2024-09-13T17:42:06.666 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T17:42:05.796138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":82696,"kb_used_data":1448,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032824,"statfs":{"total":289910292480,"available":289825611776,"internally_reserved":0,"allocated":1482752,"data_stored":1253714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.872784"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":137,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-09-13T17:41:53.903032+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:41:53.903032+0000","last_peered":"2024-09-13T17:41:53.903032+0000","last_clean":"2024-09-13T17:41:53.903032+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:41:53.903032+0000","last_undegraded":"2024-09-13T17:41:53.903032+0000","last_fullsized":"2024-09-13T17:41:53.903032+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:01.930280+0000","last_change":"2024-09-13T17:42:01.930280+0000","last_active":"2024-09-13T17:42:01.930280+0000","last_peered":"2024-09-13T17:42:01.930280+0000","last_clean":"2024-09-13T17:42:01.930280+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:01.930280+0000","last_undegraded":"2024-09-13T17:42:01.930280+0000","last_fullsized":"2024-09-13T17:42:01.930280+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-13T17:42:00.918750+0000","last_change":"2024-09-13T17:42:00.918750+0000","last_active":"2024-09-13T17:42:00.918750+0000","last_peered":"2024-09-13T17:42:00.918750+0000","last_clean":"2024-09-13T17:42:00.918750+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:00.918750+0000","last_undegraded":"2024-09-13T17:42:00.918750+0000","last_fullsized":"2024-09-13T17:42:00.918750+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":24326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":614694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:09.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:09.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.801+0000 7fbe36373640 1 Processor -- start 2024-09-13T17:42:09.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.801+0000 7fbe36373640 1 -- start start 2024-09-13T17:42:09.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe3009dc20 0x7fbe301351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3006e130 con 0x7fbe3006f6d0 2024-09-13T17:42:09.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe300a6750 con 0x7fbe3009dc20 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe3009dc20 0x7fbe301351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe3009dc20 0x7fbe301351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50542/0 (socket says 172.21.15.96:50542) 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbe3006f6d0 0x7fbe30134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33700/0 (socket says 172.21.15.96:33700) 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 -- 172.21.15.96:0/114552585 learned_addr learned my addr 172.21.15.96:0/114552585 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 -- 172.21.15.96:0/114552585 >> v1:172.21.15.96:6789/0 conn(0x7fbe3006f6d0 legacy=0x7fbe30134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2effd640 1 -- 172.21.15.96:0/114552585 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624087688 0 0) 0x7fbe3006e130 con 0x7fbe3006f6d0 2024-09-13T17:42:09.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 -- 172.21.15.96:0/114552585 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe300712e0 con 0x7fbe3009dc20 2024-09-13T17:42:09.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2f7fe640 1 --2- 172.21.15.96:0/114552585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3009dc20 0x7fbe301351c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097f0 tx=0x7fbe1801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ff0bbdc8bc0893 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2effd640 1 -- 172.21.15.96:0/114552585 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe180ab400 con 0x7fbe3009dc20 2024-09-13T17:42:09.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2effd640 1 -- 172.21.15.96:0/114552585 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe180a7050 con 0x7fbe3009dc20 2024-09-13T17:42:09.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe2effd640 1 -- 172.21.15.96:0/114552585 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe180ab6b0 con 0x7fbe3009dc20 2024-09-13T17:42:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- 172.21.15.96:0/114552585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3009dc20 msgr2=0x7fbe301351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 --2- 172.21.15.96:0/114552585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3009dc20 0x7fbe301351c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbe180097f0 tx=0x7fbe1801c450 comp rx=0 tx=0).stop 2024-09-13T17:42:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- 172.21.15.96:0/114552585 shutdown_connections 2024-09-13T17:42:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 --2- 172.21.15.96:0/114552585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3009dc20 0x7fbe301351c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- 172.21.15.96:0/114552585 >> 172.21.15.96:0/114552585 conn(0x7fbe30098a60 msgr2=0x7fbe3009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- 172.21.15.96:0/114552585 shutdown_connections 2024-09-13T17:42:09.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.805+0000 7fbe36373640 1 -- 172.21.15.96:0/114552585 wait complete. 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 Processor -- start 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 -- start start 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3009a270 con 0x7fbe3006f6d0 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50544/0 (socket says 172.21.15.96:50544) 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2ffff640 1 -- 172.21.15.96:0/255920608 learned_addr learned my addr 172.21.15.96:0/255920608 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2ffff640 1 -- 172.21.15.96:0/255920608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180094d0 con 0x7fbe3006f6d0 2024-09-13T17:42:09.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2ffff640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c420 tx=0x7fbe2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe280074d0 con 0x7fbe3006f6d0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28007630 con 0x7fbe3006f6d0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe30099cb0 con 0x7fbe3006f6d0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe28004960 con 0x7fbe3006f6d0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe300a4550 con 0x7fbe3006f6d0 2024-09-13T17:42:09.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fbe28004ac0 con 0x7fbe3006f6d0 2024-09-13T17:42:09.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 0x7fbe10037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:09.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fbe28046c60 con 0x7fbe3006f6d0 2024-09-13T17:42:09.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3013ac90 con 0x7fbe3006f6d0 2024-09-13T17:42:09.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2f7fe640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 0x7fbe10037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:09.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.809+0000 7fbe2f7fe640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 0x7fbe10037910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ad000 tx=0x7fbe18008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:09.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.813+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe2801acd0 con 0x7fbe3006f6d0 2024-09-13T17:42:09.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.969+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe300a3aa0 con 0x7fbe10035450 2024-09-13T17:42:09.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.977+0000 7fbe2cff9640 1 -- 172.21.15.96:0/255920608 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247374 (secure 0 0 0) 0x7fbe180ab760 con 0x7fbe10035450 2024-09-13T17:42:09.981 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:09.983 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 msgr2=0x7fbe10037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 0x7fbe10037910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbe180ad000 tx=0x7fbe18008040 comp rx=0 tx=0).stop 2024-09-13T17:42:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 msgr2=0x7fbe300a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c420 tx=0x7fbe2800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 shutdown_connections 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbe10035450 0x7fbe10037910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 --2- 172.21.15.96:0/255920608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe3006f6d0 0x7fbe300a5b80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 >> 172.21.15.96:0/255920608 conn(0x7fbe30098a60 msgr2=0x7fbe30139100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 shutdown_connections 2024-09-13T17:42:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:09.981+0000 7fbe36373640 1 -- 172.21.15.96:0/255920608 wait complete. 2024-09-13T17:42:09.999 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-13T17:42:09.797427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":181,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82668,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032852,"statfs":{"total":289910292480,"available":289825640448,"internally_reserved":0,"allocated":1646592,"data_stored":1306459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.862935"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322958+0000","last_change":"2024-09-13T17:42:02.360541+0000","last_active":"2024-09-13T17:42:03.322958+0000","last_peered":"2024-09-13T17:42:03.322958+0000","last_clean":"2024-09-13T17:42:03.322958+0000","last_became_active":"2024-09-13T17:42:02.359334+0000","last_became_peered":"2024-09-13T17:42:02.359334+0000","last_unstale":"2024-09-13T17:42:03.322958+0000","last_undegraded":"2024-09-13T17:42:03.322958+0000","last_fullsized":"2024-09-13T17:42:03.322958+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:42.629895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264089+0000","last_change":"2024-09-13T17:42:02.357141+0000","last_active":"2024-09-13T17:42:04.264089+0000","last_peered":"2024-09-13T17:42:04.264089+0000","last_clean":"2024-09-13T17:42:04.264089+0000","last_became_active":"2024-09-13T17:42:02.356712+0000","last_became_peered":"2024-09-13T17:42:02.356712+0000","last_unstale":"2024-09-13T17:42:04.264089+0000","last_undegraded":"2024-09-13T17:42:04.264089+0000","last_fullsized":"2024-09-13T17:42:04.264089+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322398+0000","last_change":"2024-09-13T17:42:02.353311+0000","last_active":"2024-09-13T17:42:03.322398+0000","last_peered":"2024-09-13T17:42:03.322398+0000","last_clean":"2024-09-13T17:42:03.322398+0000","last_became_active":"2024-09-13T17:42:02.351787+0000","last_became_peered":"2024-09-13T17:42:02.351787+0000","last_unstale":"2024-09-13T17:42:03.322398+0000","last_undegraded":"2024-09-13T17:42:03.322398+0000","last_fullsized":"2024-09-13T17:42:03.322398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:35.554439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912292+0000","last_change":"2024-09-13T17:42:02.364812+0000","last_active":"2024-09-13T17:42:03.912292+0000","last_peered":"2024-09-13T17:42:03.912292+0000","last_clean":"2024-09-13T17:42:03.912292+0000","last_became_active":"2024-09-13T17:42:02.364256+0000","last_became_peered":"2024-09-13T17:42:02.364256+0000","last_unstale":"2024-09-13T17:42:03.912292+0000","last_undegraded":"2024-09-13T17:42:03.912292+0000","last_fullsized":"2024-09-13T17:42:03.912292+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322703+0000","last_change":"2024-09-13T17:42:02.359338+0000","last_active":"2024-09-13T17:42:03.322703+0000","last_peered":"2024-09-13T17:42:03.322703+0000","last_clean":"2024-09-13T17:42:03.322703+0000","last_became_active":"2024-09-13T17:42:02.355220+0000","last_became_peered":"2024-09-13T17:42:02.355220+0000","last_unstale":"2024-09-13T17:42:03.322703+0000","last_undegraded":"2024-09-13T17:42:03.322703+0000","last_fullsized":"2024-09-13T17:42:03.322703+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:41.277636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911521+0000","last_change":"2024-09-13T17:42:02.364229+0000","last_active":"2024-09-13T17:42:03.911521+0000","last_peered":"2024-09-13T17:42:03.911521+0000","last_clean":"2024-09-13T17:42:03.911521+0000","last_became_active":"2024-09-13T17:42:02.363257+0000","last_became_peered":"2024-09-13T17:42:02.363257+0000","last_unstale":"2024-09-13T17:42:03.911521+0000","last_undegraded":"2024-09-13T17:42:03.911521+0000","last_fullsized":"2024-09-13T17:42:03.911521+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321735+0000","last_change":"2024-09-13T17:42:02.353689+0000","last_active":"2024-09-13T17:42:03.321735+0000","last_peered":"2024-09-13T17:42:03.321735+0000","last_clean":"2024-09-13T17:42:03.321735+0000","last_became_active":"2024-09-13T17:42:02.353466+0000","last_became_peered":"2024-09-13T17:42:02.353466+0000","last_unstale":"2024-09-13T17:42:03.321735+0000","last_undegraded":"2024-09-13T17:42:03.321735+0000","last_fullsized":"2024-09-13T17:42:03.321735+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:23.813864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911081+0000","last_change":"2024-09-13T17:42:02.351016+0000","last_active":"2024-09-13T17:42:03.911081+0000","last_peered":"2024-09-13T17:42:03.911081+0000","last_clean":"2024-09-13T17:42:03.911081+0000","last_became_active":"2024-09-13T17:42:02.349745+0000","last_became_peered":"2024-09-13T17:42:02.349745+0000","last_unstale":"2024-09-13T17:42:03.911081+0000","last_undegraded":"2024-09-13T17:42:03.911081+0000","last_fullsized":"2024-09-13T17:42:03.911081+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264317+0000","last_change":"2024-09-13T17:42:02.348417+0000","last_active":"2024-09-13T17:42:04.264317+0000","last_peered":"2024-09-13T17:42:04.264317+0000","last_clean":"2024-09-13T17:42:04.264317+0000","last_became_active":"2024-09-13T17:42:02.348073+0000","last_became_peered":"2024-09-13T17:42:02.348073+0000","last_unstale":"2024-09-13T17:42:04.264317+0000","last_undegraded":"2024-09-13T17:42:04.264317+0000","last_fullsized":"2024-09-13T17:42:04.264317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322820+0000","last_change":"2024-09-13T17:42:02.348661+0000","last_active":"2024-09-13T17:42:03.322820+0000","last_peered":"2024-09-13T17:42:03.322820+0000","last_clean":"2024-09-13T17:42:03.322820+0000","last_became_active":"2024-09-13T17:42:02.348395+0000","last_became_peered":"2024-09-13T17:42:02.348395+0000","last_unstale":"2024-09-13T17:42:03.322820+0000","last_undegraded":"2024-09-13T17:42:03.322820+0000","last_fullsized":"2024-09-13T17:42:03.322820+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:44.538462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322196+0000","last_change":"2024-09-13T17:42:02.355321+0000","last_active":"2024-09-13T17:42:03.322196+0000","last_peered":"2024-09-13T17:42:03.322196+0000","last_clean":"2024-09-13T17:42:03.322196+0000","last_became_active":"2024-09-13T17:42:02.355024+0000","last_became_peered":"2024-09-13T17:42:02.355024+0000","last_unstale":"2024-09-13T17:42:03.322196+0000","last_undegraded":"2024-09-13T17:42:03.322196+0000","last_fullsized":"2024-09-13T17:42:03.322196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:24.061971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321999+0000","last_change":"2024-09-13T17:42:02.360863+0000","last_active":"2024-09-13T17:42:03.321999+0000","last_peered":"2024-09-13T17:42:03.321999+0000","last_clean":"2024-09-13T17:42:03.321999+0000","last_became_active":"2024-09-13T17:42:02.359969+0000","last_became_peered":"2024-09-13T17:42:02.359969+0000","last_unstale":"2024-09-13T17:42:03.321999+0000","last_undegraded":"2024-09-13T17:42:03.321999+0000","last_fullsized":"2024-09-13T17:42:03.321999+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322818+0000","last_change":"2024-09-13T17:42:02.357373+0000","last_active":"2024-09-13T17:42:03.322818+0000","last_peered":"2024-09-13T17:42:03.322818+0000","last_clean":"2024-09-13T17:42:03.322818+0000","last_became_active":"2024-09-13T17:42:02.354117+0000","last_became_peered":"2024-09-13T17:42:02.354117+0000","last_unstale":"2024-09-13T17:42:03.322818+0000","last_undegraded":"2024-09-13T17:42:03.322818+0000","last_fullsized":"2024-09-13T17:42:03.322818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:28.692862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321544+0000","last_change":"2024-09-13T17:42:02.359487+0000","last_active":"2024-09-13T17:42:03.321544+0000","last_peered":"2024-09-13T17:42:03.321544+0000","last_clean":"2024-09-13T17:42:03.321544+0000","last_became_active":"2024-09-13T17:42:02.358942+0000","last_became_peered":"2024-09-13T17:42:02.358942+0000","last_unstale":"2024-09-13T17:42:03.321544+0000","last_undegraded":"2024-09-13T17:42:03.321544+0000","last_fullsized":"2024-09-13T17:42:03.321544+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:25.373355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263024+0000","last_change":"2024-09-13T17:42:02.349044+0000","last_active":"2024-09-13T17:42:04.263024+0000","last_peered":"2024-09-13T17:42:04.263024+0000","last_clean":"2024-09-13T17:42:04.263024+0000","last_became_active":"2024-09-13T17:42:02.345647+0000","last_became_peered":"2024-09-13T17:42:02.345647+0000","last_unstale":"2024-09-13T17:42:04.263024+0000","last_undegraded":"2024-09-13T17:42:04.263024+0000","last_fullsized":"2024-09-13T17:42:04.263024+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911223+0000","last_change":"2024-09-13T17:42:02.363617+0000","last_active":"2024-09-13T17:42:03.911223+0000","last_peered":"2024-09-13T17:42:03.911223+0000","last_clean":"2024-09-13T17:42:03.911223+0000","last_became_active":"2024-09-13T17:42:02.362957+0000","last_became_peered":"2024-09-13T17:42:02.362957+0000","last_unstale":"2024-09-13T17:42:03.911223+0000","last_undegraded":"2024-09-13T17:42:03.911223+0000","last_fullsized":"2024-09-13T17:42:03.911223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264682+0000","last_change":"2024-09-13T17:42:02.349237+0000","last_active":"2024-09-13T17:42:04.264682+0000","last_peered":"2024-09-13T17:42:04.264682+0000","last_clean":"2024-09-13T17:42:04.264682+0000","last_became_active":"2024-09-13T17:42:02.348179+0000","last_became_peered":"2024-09-13T17:42:02.348179+0000","last_unstale":"2024-09-13T17:42:04.264682+0000","last_undegraded":"2024-09-13T17:42:04.264682+0000","last_fullsized":"2024-09-13T17:42:04.264682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912431+0000","last_change":"2024-09-13T17:42:02.363954+0000","last_active":"2024-09-13T17:42:03.912431+0000","last_peered":"2024-09-13T17:42:03.912431+0000","last_clean":"2024-09-13T17:42:03.912431+0000","last_became_active":"2024-09-13T17:42:02.363490+0000","last_became_peered":"2024-09-13T17:42:02.363490+0000","last_unstale":"2024-09-13T17:42:03.912431+0000","last_undegraded":"2024-09-13T17:42:03.912431+0000","last_fullsized":"2024-09-13T17:42:03.912431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322498+0000","last_change":"2024-09-13T17:42:02.344432+0000","last_active":"2024-09-13T17:42:03.322498+0000","last_peered":"2024-09-13T17:42:03.322498+0000","last_clean":"2024-09-13T17:42:03.322498+0000","last_became_active":"2024-09-13T17:42:02.344155+0000","last_became_peered":"2024-09-13T17:42:02.344155+0000","last_unstale":"2024-09-13T17:42:03.322498+0000","last_undegraded":"2024-09-13T17:42:03.322498+0000","last_fullsized":"2024-09-13T17:42:03.322498+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:24.986220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321991+0000","last_change":"2024-09-13T17:42:02.360682+0000","last_active":"2024-09-13T17:42:03.321991+0000","last_peered":"2024-09-13T17:42:03.321991+0000","last_clean":"2024-09-13T17:42:03.321991+0000","last_became_active":"2024-09-13T17:42:02.356733+0000","last_became_peered":"2024-09-13T17:42:02.356733+0000","last_unstale":"2024-09-13T17:42:03.321991+0000","last_undegraded":"2024-09-13T17:42:03.321991+0000","last_fullsized":"2024-09-13T17:42:03.321991+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:43.147144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321984+0000","last_change":"2024-09-13T17:42:02.360599+0000","last_active":"2024-09-13T17:42:03.321984+0000","last_peered":"2024-09-13T17:42:03.321984+0000","last_clean":"2024-09-13T17:42:03.321984+0000","last_became_active":"2024-09-13T17:42:02.357562+0000","last_became_peered":"2024-09-13T17:42:02.357562+0000","last_unstale":"2024-09-13T17:42:03.321984+0000","last_undegraded":"2024-09-13T17:42:03.321984+0000","last_fullsized":"2024-09-13T17:42:03.321984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:41.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262849+0000","last_change":"2024-09-13T17:42:02.347884+0000","last_active":"2024-09-13T17:42:04.262849+0000","last_peered":"2024-09-13T17:42:04.262849+0000","last_clean":"2024-09-13T17:42:04.262849+0000","last_became_active":"2024-09-13T17:42:02.347662+0000","last_became_peered":"2024-09-13T17:42:02.347662+0000","last_unstale":"2024-09-13T17:42:04.262849+0000","last_undegraded":"2024-09-13T17:42:04.262849+0000","last_fullsized":"2024-09-13T17:42:04.262849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:12.394146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321378+0000","last_change":"2024-09-13T17:42:02.362310+0000","last_active":"2024-09-13T17:42:03.321378+0000","last_peered":"2024-09-13T17:42:03.321378+0000","last_clean":"2024-09-13T17:42:03.321378+0000","last_became_active":"2024-09-13T17:42:02.360999+0000","last_became_peered":"2024-09-13T17:42:02.360999+0000","last_unstale":"2024-09-13T17:42:03.321378+0000","last_undegraded":"2024-09-13T17:42:03.321378+0000","last_fullsized":"2024-09-13T17:42:03.321378+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:11.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321607+0000","last_change":"2024-09-13T17:42:02.344968+0000","last_active":"2024-09-13T17:42:03.321607+0000","last_peered":"2024-09-13T17:42:03.321607+0000","last_clean":"2024-09-13T17:42:03.321607+0000","last_became_active":"2024-09-13T17:42:02.344714+0000","last_became_peered":"2024-09-13T17:42:02.344714+0000","last_unstale":"2024-09-13T17:42:03.321607+0000","last_undegraded":"2024-09-13T17:42:03.321607+0000","last_fullsized":"2024-09-13T17:42:03.321607+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912231+0000","last_change":"2024-09-13T17:42:02.345753+0000","last_active":"2024-09-13T17:42:03.912231+0000","last_peered":"2024-09-13T17:42:03.912231+0000","last_clean":"2024-09-13T17:42:03.912231+0000","last_became_active":"2024-09-13T17:42:02.345469+0000","last_became_peered":"2024-09-13T17:42:02.345469+0000","last_unstale":"2024-09-13T17:42:03.912231+0000","last_undegraded":"2024-09-13T17:42:03.912231+0000","last_fullsized":"2024-09-13T17:42:03.912231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912733+0000","last_change":"2024-09-13T17:42:02.363798+0000","last_active":"2024-09-13T17:42:03.912733+0000","last_peered":"2024-09-13T17:42:03.912733+0000","last_clean":"2024-09-13T17:42:03.912733+0000","last_became_active":"2024-09-13T17:42:02.363138+0000","last_became_peered":"2024-09-13T17:42:02.363138+0000","last_unstale":"2024-09-13T17:42:03.912733+0000","last_undegraded":"2024-09-13T17:42:03.912733+0000","last_fullsized":"2024-09-13T17:42:03.912733+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263425+0000","last_change":"2024-09-13T17:42:02.333530+0000","last_active":"2024-09-13T17:42:04.263425+0000","last_peered":"2024-09-13T17:42:04.263425+0000","last_clean":"2024-09-13T17:42:04.263425+0000","last_became_active":"2024-09-13T17:42:02.329725+0000","last_became_peered":"2024-09-13T17:42:02.329725+0000","last_unstale":"2024-09-13T17:42:04.263425+0000","last_undegraded":"2024-09-13T17:42:04.263425+0000","last_fullsized":"2024-09-13T17:42:04.263425+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322104+0000","last_change":"2024-09-13T17:42:02.360444+0000","last_active":"2024-09-13T17:42:03.322104+0000","last_peered":"2024-09-13T17:42:03.322104+0000","last_clean":"2024-09-13T17:42:03.322104+0000","last_became_active":"2024-09-13T17:42:02.357480+0000","last_became_peered":"2024-09-13T17:42:02.357480+0000","last_unstale":"2024-09-13T17:42:03.322104+0000","last_undegraded":"2024-09-13T17:42:03.322104+0000","last_fullsized":"2024-09-13T17:42:03.322104+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:18.088198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322073+0000","last_change":"2024-09-13T17:42:02.360990+0000","last_active":"2024-09-13T17:42:03.322073+0000","last_peered":"2024-09-13T17:42:03.322073+0000","last_clean":"2024-09-13T17:42:03.322073+0000","last_became_active":"2024-09-13T17:42:02.356745+0000","last_became_peered":"2024-09-13T17:42:02.356745+0000","last_unstale":"2024-09-13T17:42:03.322073+0000","last_undegraded":"2024-09-13T17:42:03.322073+0000","last_fullsized":"2024-09-13T17:42:03.322073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321744+0000","last_change":"2024-09-13T17:42:02.351170+0000","last_active":"2024-09-13T17:42:03.321744+0000","last_peered":"2024-09-13T17:42:03.321744+0000","last_clean":"2024-09-13T17:42:03.321744+0000","last_became_active":"2024-09-13T17:42:02.350917+0000","last_became_peered":"2024-09-13T17:42:02.350917+0000","last_unstale":"2024-09-13T17:42:03.321744+0000","last_undegraded":"2024-09-13T17:42:03.321744+0000","last_fullsized":"2024-09-13T17:42:03.321744+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.975775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912828+0000","last_change":"2024-09-13T17:42:02.337517+0000","last_active":"2024-09-13T17:42:03.912828+0000","last_peered":"2024-09-13T17:42:03.912828+0000","last_clean":"2024-09-13T17:42:03.912828+0000","last_became_active":"2024-09-13T17:42:02.336903+0000","last_became_peered":"2024-09-13T17:42:02.336903+0000","last_unstale":"2024-09-13T17:42:03.912828+0000","last_undegraded":"2024-09-13T17:42:03.912828+0000","last_fullsized":"2024-09-13T17:42:03.912828+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:46.917681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912186+0000","last_change":"2024-09-13T17:42:02.364079+0000","last_active":"2024-09-13T17:42:03.912186+0000","last_peered":"2024-09-13T17:42:03.912186+0000","last_clean":"2024-09-13T17:42:03.912186+0000","last_became_active":"2024-09-13T17:42:02.363628+0000","last_became_peered":"2024-09-13T17:42:02.363628+0000","last_unstale":"2024-09-13T17:42:03.912186+0000","last_undegraded":"2024-09-13T17:42:03.912186+0000","last_fullsized":"2024-09-13T17:42:03.912186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263255+0000","last_change":"2024-09-13T17:42:02.333698+0000","last_active":"2024-09-13T17:42:04.263255+0000","last_peered":"2024-09-13T17:42:04.263255+0000","last_clean":"2024-09-13T17:42:04.263255+0000","last_became_active":"2024-09-13T17:42:02.329909+0000","last_became_peered":"2024-09-13T17:42:02.329909+0000","last_unstale":"2024-09-13T17:42:04.263255+0000","last_undegraded":"2024-09-13T17:42:04.263255+0000","last_fullsized":"2024-09-13T17:42:04.263255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264107+0000","last_change":"2024-09-13T17:42:01.989110+0000","last_active":"2024-09-13T17:42:04.264107+0000","last_peered":"2024-09-13T17:42:04.264107+0000","last_clean":"2024-09-13T17:42:04.264107+0000","last_became_active":"2024-09-13T17:42:01.988776+0000","last_became_peered":"2024-09-13T17:42:01.988776+0000","last_unstale":"2024-09-13T17:42:04.264107+0000","last_undegraded":"2024-09-13T17:42:04.264107+0000","last_fullsized":"2024-09-13T17:42:04.264107+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322159+0000","last_change":"2024-09-13T17:42:02.360143+0000","last_active":"2024-09-13T17:42:03.322159+0000","last_peered":"2024-09-13T17:42:03.322159+0000","last_clean":"2024-09-13T17:42:03.322159+0000","last_became_active":"2024-09-13T17:42:02.357112+0000","last_became_peered":"2024-09-13T17:42:02.357112+0000","last_unstale":"2024-09-13T17:42:03.322159+0000","last_undegraded":"2024-09-13T17:42:03.322159+0000","last_fullsized":"2024-09-13T17:42:03.322159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:53.057536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911782+0000","last_change":"2024-09-13T17:42:01.989114+0000","last_active":"2024-09-13T17:42:03.911782+0000","last_peered":"2024-09-13T17:42:03.911782+0000","last_clean":"2024-09-13T17:42:03.911782+0000","last_became_active":"2024-09-13T17:42:01.988536+0000","last_became_peered":"2024-09-13T17:42:01.988536+0000","last_unstale":"2024-09-13T17:42:03.911782+0000","last_undegraded":"2024-09-13T17:42:03.911782+0000","last_fullsized":"2024-09-13T17:42:03.911782+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321624+0000","last_change":"2024-09-13T17:42:01.990295+0000","last_active":"2024-09-13T17:42:03.321624+0000","last_peered":"2024-09-13T17:42:03.321624+0000","last_clean":"2024-09-13T17:42:03.321624+0000","last_became_active":"2024-09-13T17:42:01.989951+0000","last_became_peered":"2024-09-13T17:42:01.989951+0000","last_unstale":"2024-09-13T17:42:03.321624+0000","last_undegraded":"2024-09-13T17:42:03.321624+0000","last_fullsized":"2024-09-13T17:42:03.321624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911808+0000","last_change":"2024-09-13T17:42:02.363919+0000","last_active":"2024-09-13T17:42:03.911808+0000","last_peered":"2024-09-13T17:42:03.911808+0000","last_clean":"2024-09-13T17:42:03.911808+0000","last_became_active":"2024-09-13T17:42:02.363337+0000","last_became_peered":"2024-09-13T17:42:02.363337+0000","last_unstale":"2024-09-13T17:42:03.911808+0000","last_undegraded":"2024-09-13T17:42:03.911808+0000","last_fullsized":"2024-09-13T17:42:03.911808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321842+0000","last_change":"2024-09-13T17:42:01.977030+0000","last_active":"2024-09-13T17:42:03.321842+0000","last_peered":"2024-09-13T17:42:03.321842+0000","last_clean":"2024-09-13T17:42:03.321842+0000","last_became_active":"2024-09-13T17:42:01.976768+0000","last_became_peered":"2024-09-13T17:42:01.976768+0000","last_unstale":"2024-09-13T17:42:03.321842+0000","last_undegraded":"2024-09-13T17:42:03.321842+0000","last_fullsized":"2024-09-13T17:42:03.321842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262490+0000","last_change":"2024-09-13T17:42:02.329537+0000","last_active":"2024-09-13T17:42:04.262490+0000","last_peered":"2024-09-13T17:42:04.262490+0000","last_clean":"2024-09-13T17:42:04.262490+0000","last_became_active":"2024-09-13T17:42:02.329246+0000","last_became_peered":"2024-09-13T17:42:02.329246+0000","last_unstale":"2024-09-13T17:42:04.262490+0000","last_undegraded":"2024-09-13T17:42:04.262490+0000","last_fullsized":"2024-09-13T17:42:04.262490+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264525+0000","last_change":"2024-09-13T17:42:01.974367+0000","last_active":"2024-09-13T17:42:04.264525+0000","last_peered":"2024-09-13T17:42:04.264525+0000","last_clean":"2024-09-13T17:42:04.264525+0000","last_became_active":"2024-09-13T17:42:01.974054+0000","last_became_peered":"2024-09-13T17:42:01.974054+0000","last_unstale":"2024-09-13T17:42:04.264525+0000","last_undegraded":"2024-09-13T17:42:04.264525+0000","last_fullsized":"2024-09-13T17:42:04.264525+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912913+0000","last_change":"2024-09-13T17:42:02.341032+0000","last_active":"2024-09-13T17:42:03.912913+0000","last_peered":"2024-09-13T17:42:03.912913+0000","last_clean":"2024-09-13T17:42:03.912913+0000","last_became_active":"2024-09-13T17:42:02.340777+0000","last_became_peered":"2024-09-13T17:42:02.340777+0000","last_unstale":"2024-09-13T17:42:03.912913+0000","last_undegraded":"2024-09-13T17:42:03.912913+0000","last_fullsized":"2024-09-13T17:42:03.912913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:37.537177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.973354+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.969555+0000","last_became_peered":"2024-09-13T17:42:01.969555+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323123+0000","last_change":"2024-09-13T17:42:02.356982+0000","last_active":"2024-09-13T17:42:03.323123+0000","last_peered":"2024-09-13T17:42:03.323123+0000","last_clean":"2024-09-13T17:42:03.323123+0000","last_became_active":"2024-09-13T17:42:02.354879+0000","last_became_peered":"2024-09-13T17:42:02.354879+0000","last_unstale":"2024-09-13T17:42:03.323123+0000","last_undegraded":"2024-09-13T17:42:03.323123+0000","last_fullsized":"2024-09-13T17:42:03.323123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:00.269193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264141+0000","last_change":"2024-09-13T17:42:01.988113+0000","last_active":"2024-09-13T17:42:04.264141+0000","last_peered":"2024-09-13T17:42:04.264141+0000","last_clean":"2024-09-13T17:42:04.264141+0000","last_became_active":"2024-09-13T17:42:01.987821+0000","last_became_peered":"2024-09-13T17:42:01.987821+0000","last_unstale":"2024-09-13T17:42:04.264141+0000","last_undegraded":"2024-09-13T17:42:04.264141+0000","last_fullsized":"2024-09-13T17:42:04.264141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263852+0000","last_change":"2024-09-13T17:42:02.342353+0000","last_active":"2024-09-13T17:42:04.263852+0000","last_peered":"2024-09-13T17:42:04.263852+0000","last_clean":"2024-09-13T17:42:04.263852+0000","last_became_active":"2024-09-13T17:42:02.336419+0000","last_became_peered":"2024-09-13T17:42:02.336419+0000","last_unstale":"2024-09-13T17:42:04.263852+0000","last_undegraded":"2024-09-13T17:42:04.263852+0000","last_fullsized":"2024-09-13T17:42:04.263852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912570+0000","last_change":"2024-09-13T17:42:01.981005+0000","last_active":"2024-09-13T17:42:03.912570+0000","last_peered":"2024-09-13T17:42:03.912570+0000","last_clean":"2024-09-13T17:42:03.912570+0000","last_became_active":"2024-09-13T17:42:01.978436+0000","last_became_peered":"2024-09-13T17:42:01.978436+0000","last_unstale":"2024-09-13T17:42:03.912570+0000","last_undegraded":"2024-09-13T17:42:03.912570+0000","last_fullsized":"2024-09-13T17:42:03.912570+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322389+0000","last_change":"2024-09-13T17:42:02.354827+0000","last_active":"2024-09-13T17:42:03.322389+0000","last_peered":"2024-09-13T17:42:03.322389+0000","last_clean":"2024-09-13T17:42:03.322389+0000","last_became_active":"2024-09-13T17:42:02.353803+0000","last_became_peered":"2024-09-13T17:42:02.353803+0000","last_unstale":"2024-09-13T17:42:03.322389+0000","last_undegraded":"2024-09-13T17:42:03.322389+0000","last_fullsized":"2024-09-13T17:42:03.322389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:17.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322676+0000","last_change":"2024-09-13T17:42:01.981800+0000","last_active":"2024-09-13T17:42:03.322676+0000","last_peered":"2024-09-13T17:42:03.322676+0000","last_clean":"2024-09-13T17:42:03.322676+0000","last_became_active":"2024-09-13T17:42:01.978579+0000","last_became_peered":"2024-09-13T17:42:01.978579+0000","last_unstale":"2024-09-13T17:42:03.322676+0000","last_undegraded":"2024-09-13T17:42:03.322676+0000","last_fullsized":"2024-09-13T17:42:03.322676+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321641+0000","last_change":"2024-09-13T17:42:02.360084+0000","last_active":"2024-09-13T17:42:03.321641+0000","last_peered":"2024-09-13T17:42:03.321641+0000","last_clean":"2024-09-13T17:42:03.321641+0000","last_became_active":"2024-09-13T17:42:02.358943+0000","last_became_peered":"2024-09-13T17:42:02.358943+0000","last_unstale":"2024-09-13T17:42:03.321641+0000","last_undegraded":"2024-09-13T17:42:03.321641+0000","last_fullsized":"2024-09-13T17:42:03.321641+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:30.309069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911963+0000","last_change":"2024-09-13T17:42:01.980802+0000","last_active":"2024-09-13T17:42:03.911963+0000","last_peered":"2024-09-13T17:42:03.911963+0000","last_clean":"2024-09-13T17:42:03.911963+0000","last_became_active":"2024-09-13T17:42:01.977408+0000","last_became_peered":"2024-09-13T17:42:01.977408+0000","last_unstale":"2024-09-13T17:42:03.911963+0000","last_undegraded":"2024-09-13T17:42:03.911963+0000","last_fullsized":"2024-09-13T17:42:03.911963+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:21.671479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263263+0000","last_change":"2024-09-13T17:42:02.356627+0000","last_active":"2024-09-13T17:42:04.263263+0000","last_peered":"2024-09-13T17:42:04.263263+0000","last_clean":"2024-09-13T17:42:04.263263+0000","last_became_active":"2024-09-13T17:42:02.356381+0000","last_became_peered":"2024-09-13T17:42:02.356381+0000","last_unstale":"2024-09-13T17:42:04.263263+0000","last_undegraded":"2024-09-13T17:42:04.263263+0000","last_fullsized":"2024-09-13T17:42:04.263263+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321850+0000","last_change":"2024-09-13T17:42:01.973218+0000","last_active":"2024-09-13T17:42:03.321850+0000","last_peered":"2024-09-13T17:42:03.321850+0000","last_clean":"2024-09-13T17:42:03.321850+0000","last_became_active":"2024-09-13T17:42:01.972688+0000","last_became_peered":"2024-09-13T17:42:01.972688+0000","last_unstale":"2024-09-13T17:42:03.321850+0000","last_undegraded":"2024-09-13T17:42:03.321850+0000","last_fullsized":"2024-09-13T17:42:03.321850+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262433+0000","last_change":"2024-09-13T17:42:02.348071+0000","last_active":"2024-09-13T17:42:04.262433+0000","last_peered":"2024-09-13T17:42:04.262433+0000","last_clean":"2024-09-13T17:42:04.262433+0000","last_became_active":"2024-09-13T17:42:02.347463+0000","last_became_peered":"2024-09-13T17:42:02.347463+0000","last_unstale":"2024-09-13T17:42:04.262433+0000","last_undegraded":"2024-09-13T17:42:04.262433+0000","last_fullsized":"2024-09-13T17:42:04.262433+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912464+0000","last_change":"2024-09-13T17:42:01.981250+0000","last_active":"2024-09-13T17:42:03.912464+0000","last_peered":"2024-09-13T17:42:03.912464+0000","last_clean":"2024-09-13T17:42:03.912464+0000","last_became_active":"2024-09-13T17:42:01.978674+0000","last_became_peered":"2024-09-13T17:42:01.978674+0000","last_unstale":"2024-09-13T17:42:03.912464+0000","last_undegraded":"2024-09-13T17:42:03.912464+0000","last_fullsized":"2024-09-13T17:42:03.912464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911530+0000","last_change":"2024-09-13T17:42:02.339567+0000","last_active":"2024-09-13T17:42:03.911530+0000","last_peered":"2024-09-13T17:42:03.911530+0000","last_clean":"2024-09-13T17:42:03.911530+0000","last_became_active":"2024-09-13T17:42:02.339304+0000","last_became_peered":"2024-09-13T17:42:02.339304+0000","last_unstale":"2024-09-13T17:42:03.911530+0000","last_undegraded":"2024-09-13T17:42:03.911530+0000","last_fullsized":"2024-09-13T17:42:03.911530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:05:01.291734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323607+0000","last_change":"2024-09-13T17:42:01.973755+0000","last_active":"2024-09-13T17:42:03.323607+0000","last_peered":"2024-09-13T17:42:03.323607+0000","last_clean":"2024-09-13T17:42:03.323607+0000","last_became_active":"2024-09-13T17:42:01.973179+0000","last_became_peered":"2024-09-13T17:42:01.973179+0000","last_unstale":"2024-09-13T17:42:03.323607+0000","last_undegraded":"2024-09-13T17:42:03.323607+0000","last_fullsized":"2024-09-13T17:42:03.323607+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912974+0000","last_change":"2024-09-13T17:42:02.363936+0000","last_active":"2024-09-13T17:42:03.912974+0000","last_peered":"2024-09-13T17:42:03.912974+0000","last_clean":"2024-09-13T17:42:03.912974+0000","last_became_active":"2024-09-13T17:42:02.363316+0000","last_became_peered":"2024-09-13T17:42:02.363316+0000","last_unstale":"2024-09-13T17:42:03.912974+0000","last_undegraded":"2024-09-13T17:42:03.912974+0000","last_fullsized":"2024-09-13T17:42:03.912974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913097+0000","last_change":"2024-09-13T17:42:01.987237+0000","last_active":"2024-09-13T17:42:03.913097+0000","last_peered":"2024-09-13T17:42:03.913097+0000","last_clean":"2024-09-13T17:42:03.913097+0000","last_became_active":"2024-09-13T17:42:01.986684+0000","last_became_peered":"2024-09-13T17:42:01.986684+0000","last_unstale":"2024-09-13T17:42:03.913097+0000","last_undegraded":"2024-09-13T17:42:03.913097+0000","last_fullsized":"2024-09-13T17:42:03.913097+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:27.554217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323169+0000","last_change":"2024-09-13T17:42:02.359350+0000","last_active":"2024-09-13T17:42:03.323169+0000","last_peered":"2024-09-13T17:42:03.323169+0000","last_clean":"2024-09-13T17:42:03.323169+0000","last_became_active":"2024-09-13T17:42:02.357233+0000","last_became_peered":"2024-09-13T17:42:02.357233+0000","last_unstale":"2024-09-13T17:42:03.323169+0000","last_undegraded":"2024-09-13T17:42:03.323169+0000","last_fullsized":"2024-09-13T17:42:03.323169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.541840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264444+0000","last_change":"2024-09-13T17:42:01.970689+0000","last_active":"2024-09-13T17:42:04.264444+0000","last_peered":"2024-09-13T17:42:04.264444+0000","last_clean":"2024-09-13T17:42:04.264444+0000","last_became_active":"2024-09-13T17:42:01.970057+0000","last_became_peered":"2024-09-13T17:42:01.970057+0000","last_unstale":"2024-09-13T17:42:04.264444+0000","last_undegraded":"2024-09-13T17:42:04.264444+0000","last_fullsized":"2024-09-13T17:42:04.264444+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322140+0000","last_change":"2024-09-13T17:42:02.360241+0000","last_active":"2024-09-13T17:42:03.322140+0000","last_peered":"2024-09-13T17:42:03.322140+0000","last_clean":"2024-09-13T17:42:03.322140+0000","last_became_active":"2024-09-13T17:42:02.357150+0000","last_became_peered":"2024-09-13T17:42:02.357150+0000","last_unstale":"2024-09-13T17:42:03.322140+0000","last_undegraded":"2024-09-13T17:42:03.322140+0000","last_fullsized":"2024-09-13T17:42:03.322140+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:36:55.337229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912797+0000","last_change":"2024-09-13T17:42:01.987432+0000","last_active":"2024-09-13T17:42:03.912797+0000","last_peered":"2024-09-13T17:42:03.912797+0000","last_clean":"2024-09-13T17:42:03.912797+0000","last_became_active":"2024-09-13T17:42:01.986822+0000","last_became_peered":"2024-09-13T17:42:01.986822+0000","last_unstale":"2024-09-13T17:42:03.912797+0000","last_undegraded":"2024-09-13T17:42:03.912797+0000","last_fullsized":"2024-09-13T17:42:03.912797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:59.277624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263691+0000","last_change":"2024-09-13T17:42:02.349202+0000","last_active":"2024-09-13T17:42:04.263691+0000","last_peered":"2024-09-13T17:42:04.263691+0000","last_clean":"2024-09-13T17:42:04.263691+0000","last_became_active":"2024-09-13T17:42:02.345781+0000","last_became_peered":"2024-09-13T17:42:02.345781+0000","last_unstale":"2024-09-13T17:42:04.263691+0000","last_undegraded":"2024-09-13T17:42:04.263691+0000","last_fullsized":"2024-09-13T17:42:04.263691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":141,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.317552+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:02.317552+0000","last_peered":"2024-09-13T17:42:02.317552+0000","last_clean":"2024-09-13T17:42:02.317552+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:02.317552+0000","last_undegraded":"2024-09-13T17:42:02.317552+0000","last_fullsized":"2024-09-13T17:42:02.317552+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263958+0000","last_change":"2024-09-13T17:42:01.969286+0000","last_active":"2024-09-13T17:42:04.263958+0000","last_peered":"2024-09-13T17:42:04.263958+0000","last_clean":"2024-09-13T17:42:04.263958+0000","last_became_active":"2024-09-13T17:42:01.968921+0000","last_became_peered":"2024-09-13T17:42:01.968921+0000","last_unstale":"2024-09-13T17:42:04.263958+0000","last_undegraded":"2024-09-13T17:42:04.263958+0000","last_fullsized":"2024-09-13T17:42:04.263958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263120+0000","last_change":"2024-09-13T17:42:02.329801+0000","last_active":"2024-09-13T17:42:04.263120+0000","last_peered":"2024-09-13T17:42:04.263120+0000","last_clean":"2024-09-13T17:42:04.263120+0000","last_became_active":"2024-09-13T17:42:02.329300+0000","last_became_peered":"2024-09-13T17:42:02.329300+0000","last_unstale":"2024-09-13T17:42:04.263120+0000","last_undegraded":"2024-09-13T17:42:04.263120+0000","last_fullsized":"2024-09-13T17:42:04.263120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912706+0000","last_change":"2024-09-13T17:42:01.982284+0000","last_active":"2024-09-13T17:42:03.912706+0000","last_peered":"2024-09-13T17:42:03.912706+0000","last_clean":"2024-09-13T17:42:03.912706+0000","last_became_active":"2024-09-13T17:42:01.981495+0000","last_became_peered":"2024-09-13T17:42:01.981495+0000","last_unstale":"2024-09-13T17:42:03.912706+0000","last_undegraded":"2024-09-13T17:42:03.912706+0000","last_fullsized":"2024-09-13T17:42:03.912706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321462+0000","last_change":"2024-09-13T17:42:02.359429+0000","last_active":"2024-09-13T17:42:03.321462+0000","last_peered":"2024-09-13T17:42:03.321462+0000","last_clean":"2024-09-13T17:42:03.321462+0000","last_became_active":"2024-09-13T17:42:02.357531+0000","last_became_peered":"2024-09-13T17:42:02.357531+0000","last_unstale":"2024-09-13T17:42:03.321462+0000","last_undegraded":"2024-09-13T17:42:03.321462+0000","last_fullsized":"2024-09-13T17:42:03.321462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:43.826159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911994+0000","last_change":"2024-09-13T17:42:01.980799+0000","last_active":"2024-09-13T17:42:03.911994+0000","last_peered":"2024-09-13T17:42:03.911994+0000","last_clean":"2024-09-13T17:42:03.911994+0000","last_became_active":"2024-09-13T17:42:01.977918+0000","last_became_peered":"2024-09-13T17:42:01.977918+0000","last_unstale":"2024-09-13T17:42:03.911994+0000","last_undegraded":"2024-09-13T17:42:03.911994+0000","last_fullsized":"2024-09-13T17:42:03.911994+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263884+0000","last_change":"2024-09-13T17:42:02.339725+0000","last_active":"2024-09-13T17:42:04.263884+0000","last_peered":"2024-09-13T17:42:04.263884+0000","last_clean":"2024-09-13T17:42:04.263884+0000","last_became_active":"2024-09-13T17:42:02.334096+0000","last_became_peered":"2024-09-13T17:42:02.334096+0000","last_unstale":"2024-09-13T17:42:04.263884+0000","last_undegraded":"2024-09-13T17:42:04.263884+0000","last_fullsized":"2024-09-13T17:42:04.263884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322118+0000","last_change":"2024-09-13T17:42:01.982743+0000","last_active":"2024-09-13T17:42:03.322118+0000","last_peered":"2024-09-13T17:42:03.322118+0000","last_clean":"2024-09-13T17:42:03.322118+0000","last_became_active":"2024-09-13T17:42:01.981897+0000","last_became_peered":"2024-09-13T17:42:01.981897+0000","last_unstale":"2024-09-13T17:42:03.322118+0000","last_undegraded":"2024-09-13T17:42:03.322118+0000","last_fullsized":"2024-09-13T17:42:03.322118+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322270+0000","last_change":"2024-09-13T17:42:02.360779+0000","last_active":"2024-09-13T17:42:03.322270+0000","last_peered":"2024-09-13T17:42:03.322270+0000","last_clean":"2024-09-13T17:42:03.322270+0000","last_became_active":"2024-09-13T17:42:02.359476+0000","last_became_peered":"2024-09-13T17:42:02.359476+0000","last_unstale":"2024-09-13T17:42:03.322270+0000","last_undegraded":"2024-09-13T17:42:03.322270+0000","last_fullsized":"2024-09-13T17:42:03.322270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264113+0000","last_change":"2024-09-13T17:42:01.988534+0000","last_active":"2024-09-13T17:42:04.264113+0000","last_peered":"2024-09-13T17:42:04.264113+0000","last_clean":"2024-09-13T17:42:04.264113+0000","last_became_active":"2024-09-13T17:42:01.988254+0000","last_became_peered":"2024-09-13T17:42:01.988254+0000","last_unstale":"2024-09-13T17:42:04.264113+0000","last_undegraded":"2024-09-13T17:42:04.264113+0000","last_fullsized":"2024-09-13T17:42:04.264113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912533+0000","last_change":"2024-09-13T17:42:02.338990+0000","last_active":"2024-09-13T17:42:03.912533+0000","last_peered":"2024-09-13T17:42:03.912533+0000","last_clean":"2024-09-13T17:42:03.912533+0000","last_became_active":"2024-09-13T17:42:02.338737+0000","last_became_peered":"2024-09-13T17:42:02.338737+0000","last_unstale":"2024-09-13T17:42:03.912533+0000","last_undegraded":"2024-09-13T17:42:03.912533+0000","last_fullsized":"2024-09-13T17:42:03.912533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:46:10.901730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264464+0000","last_change":"2024-09-13T17:42:01.966772+0000","last_active":"2024-09-13T17:42:04.264464+0000","last_peered":"2024-09-13T17:42:04.264464+0000","last_clean":"2024-09-13T17:42:04.264464+0000","last_became_active":"2024-09-13T17:42:01.965762+0000","last_became_peered":"2024-09-13T17:42:01.965762+0000","last_unstale":"2024-09-13T17:42:04.264464+0000","last_undegraded":"2024-09-13T17:42:04.264464+0000","last_fullsized":"2024-09-13T17:42:04.264464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264436+0000","last_change":"2024-09-13T17:42:02.340638+0000","last_active":"2024-09-13T17:42:04.264436+0000","last_peered":"2024-09-13T17:42:04.264436+0000","last_clean":"2024-09-13T17:42:04.264436+0000","last_became_active":"2024-09-13T17:42:02.336028+0000","last_became_peered":"2024-09-13T17:42:02.336028+0000","last_unstale":"2024-09-13T17:42:04.264436+0000","last_undegraded":"2024-09-13T17:42:04.264436+0000","last_fullsized":"2024-09-13T17:42:04.264436+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264337+0000","last_change":"2024-09-13T17:42:01.969377+0000","last_active":"2024-09-13T17:42:04.264337+0000","last_peered":"2024-09-13T17:42:04.264337+0000","last_clean":"2024-09-13T17:42:04.264337+0000","last_became_active":"2024-09-13T17:42:01.966888+0000","last_became_peered":"2024-09-13T17:42:01.966888+0000","last_unstale":"2024-09-13T17:42:04.264337+0000","last_undegraded":"2024-09-13T17:42:04.264337+0000","last_fullsized":"2024-09-13T17:42:04.264337+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912530+0000","last_change":"2024-09-13T17:42:02.336164+0000","last_active":"2024-09-13T17:42:03.912530+0000","last_peered":"2024-09-13T17:42:03.912530+0000","last_clean":"2024-09-13T17:42:03.912530+0000","last_became_active":"2024-09-13T17:42:02.335891+0000","last_became_peered":"2024-09-13T17:42:02.335891+0000","last_unstale":"2024-09-13T17:42:03.912530+0000","last_undegraded":"2024-09-13T17:42:03.912530+0000","last_fullsized":"2024-09-13T17:42:03.912530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264680+0000","last_change":"2024-09-13T17:42:01.974797+0000","last_active":"2024-09-13T17:42:04.264680+0000","last_peered":"2024-09-13T17:42:04.264680+0000","last_clean":"2024-09-13T17:42:04.264680+0000","last_became_active":"2024-09-13T17:42:01.971445+0000","last_became_peered":"2024-09-13T17:42:01.971445+0000","last_unstale":"2024-09-13T17:42:04.264680+0000","last_undegraded":"2024-09-13T17:42:04.264680+0000","last_fullsized":"2024-09-13T17:42:04.264680+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322893+0000","last_change":"2024-09-13T17:42:02.354060+0000","last_active":"2024-09-13T17:42:03.322893+0000","last_peered":"2024-09-13T17:42:03.322893+0000","last_clean":"2024-09-13T17:42:03.322893+0000","last_became_active":"2024-09-13T17:42:02.353776+0000","last_became_peered":"2024-09-13T17:42:02.353776+0000","last_unstale":"2024-09-13T17:42:03.322893+0000","last_undegraded":"2024-09-13T17:42:03.322893+0000","last_fullsized":"2024-09-13T17:42:03.322893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.005234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322185+0000","last_change":"2024-09-13T17:42:01.983210+0000","last_active":"2024-09-13T17:42:03.322185+0000","last_peered":"2024-09-13T17:42:03.322185+0000","last_clean":"2024-09-13T17:42:03.322185+0000","last_became_active":"2024-09-13T17:42:01.978757+0000","last_became_peered":"2024-09-13T17:42:01.978757+0000","last_unstale":"2024-09-13T17:42:03.322185+0000","last_undegraded":"2024-09-13T17:42:03.322185+0000","last_fullsized":"2024-09-13T17:42:03.322185+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321316+0000","last_change":"2024-09-13T17:42:02.359330+0000","last_active":"2024-09-13T17:42:03.321316+0000","last_peered":"2024-09-13T17:42:03.321316+0000","last_clean":"2024-09-13T17:42:03.321316+0000","last_became_active":"2024-09-13T17:42:02.358904+0000","last_became_peered":"2024-09-13T17:42:02.358904+0000","last_unstale":"2024-09-13T17:42:03.321316+0000","last_undegraded":"2024-09-13T17:42:03.321316+0000","last_fullsized":"2024-09-13T17:42:03.321316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:45.730743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263264+0000","last_change":"2024-09-13T17:42:01.988623+0000","last_active":"2024-09-13T17:42:04.263264+0000","last_peered":"2024-09-13T17:42:04.263264+0000","last_clean":"2024-09-13T17:42:04.263264+0000","last_became_active":"2024-09-13T17:42:01.988147+0000","last_became_peered":"2024-09-13T17:42:01.988147+0000","last_unstale":"2024-09-13T17:42:04.263264+0000","last_undegraded":"2024-09-13T17:42:04.263264+0000","last_fullsized":"2024-09-13T17:42:04.263264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912143+0000","last_change":"2024-09-13T17:42:02.341209+0000","last_active":"2024-09-13T17:42:03.912143+0000","last_peered":"2024-09-13T17:42:03.912143+0000","last_clean":"2024-09-13T17:42:03.912143+0000","last_became_active":"2024-09-13T17:42:02.340934+0000","last_became_peered":"2024-09-13T17:42:02.340934+0000","last_unstale":"2024-09-13T17:42:03.912143+0000","last_undegraded":"2024-09-13T17:42:03.912143+0000","last_fullsized":"2024-09-13T17:42:03.912143+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:34:26.164972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263495+0000","last_change":"2024-09-13T17:42:01.988051+0000","last_active":"2024-09-13T17:42:04.263495+0000","last_peered":"2024-09-13T17:42:04.263495+0000","last_clean":"2024-09-13T17:42:04.263495+0000","last_became_active":"2024-09-13T17:42:01.987703+0000","last_became_peered":"2024-09-13T17:42:01.987703+0000","last_unstale":"2024-09-13T17:42:04.263495+0000","last_undegraded":"2024-09-13T17:42:04.263495+0000","last_fullsized":"2024-09-13T17:42:04.263495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263566+0000","last_change":"2024-09-13T17:42:02.347875+0000","last_active":"2024-09-13T17:42:04.263566+0000","last_peered":"2024-09-13T17:42:04.263566+0000","last_clean":"2024-09-13T17:42:04.263566+0000","last_became_active":"2024-09-13T17:42:02.347616+0000","last_became_peered":"2024-09-13T17:42:02.347616+0000","last_unstale":"2024-09-13T17:42:04.263566+0000","last_undegraded":"2024-09-13T17:42:04.263566+0000","last_fullsized":"2024-09-13T17:42:04.263566+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.988437+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.987164+0000","last_became_peered":"2024-09-13T17:42:01.987164+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322005+0000","last_change":"2024-09-13T17:42:02.355026+0000","last_active":"2024-09-13T17:42:03.322005+0000","last_peered":"2024-09-13T17:42:03.322005+0000","last_clean":"2024-09-13T17:42:03.322005+0000","last_became_active":"2024-09-13T17:42:02.353935+0000","last_became_peered":"2024-09-13T17:42:02.353935+0000","last_unstale":"2024-09-13T17:42:03.322005+0000","last_undegraded":"2024-09-13T17:42:03.322005+0000","last_fullsized":"2024-09-13T17:42:03.322005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:54.702173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912398+0000","last_change":"2024-09-13T17:42:02.363456+0000","last_active":"2024-09-13T17:42:03.912398+0000","last_peered":"2024-09-13T17:42:03.912398+0000","last_clean":"2024-09-13T17:42:03.912398+0000","last_became_active":"2024-09-13T17:42:02.363171+0000","last_became_peered":"2024-09-13T17:42:02.363171+0000","last_unstale":"2024-09-13T17:42:03.912398+0000","last_undegraded":"2024-09-13T17:42:03.912398+0000","last_fullsized":"2024-09-13T17:42:03.912398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322259+0000","last_change":"2024-09-13T17:42:01.977493+0000","last_active":"2024-09-13T17:42:03.322259+0000","last_peered":"2024-09-13T17:42:03.322259+0000","last_clean":"2024-09-13T17:42:03.322259+0000","last_became_active":"2024-09-13T17:42:01.976932+0000","last_became_peered":"2024-09-13T17:42:01.976932+0000","last_unstale":"2024-09-13T17:42:03.322259+0000","last_undegraded":"2024-09-13T17:42:03.322259+0000","last_fullsized":"2024-09-13T17:42:03.322259+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263697+0000","last_change":"2024-09-13T17:42:02.348464+0000","last_active":"2024-09-13T17:42:04.263697+0000","last_peered":"2024-09-13T17:42:04.263697+0000","last_clean":"2024-09-13T17:42:04.263697+0000","last_became_active":"2024-09-13T17:42:02.348208+0000","last_became_peered":"2024-09-13T17:42:02.348208+0000","last_unstale":"2024-09-13T17:42:04.263697+0000","last_undegraded":"2024-09-13T17:42:04.263697+0000","last_fullsized":"2024-09-13T17:42:04.263697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322576+0000","last_change":"2024-09-13T17:42:01.986020+0000","last_active":"2024-09-13T17:42:03.322576+0000","last_peered":"2024-09-13T17:42:03.322576+0000","last_clean":"2024-09-13T17:42:03.322576+0000","last_became_active":"2024-09-13T17:42:01.985663+0000","last_became_peered":"2024-09-13T17:42:01.985663+0000","last_unstale":"2024-09-13T17:42:03.322576+0000","last_undegraded":"2024-09-13T17:42:03.322576+0000","last_fullsized":"2024-09-13T17:42:03.322576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:12.942087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323065+0000","last_change":"2024-09-13T17:42:02.359950+0000","last_active":"2024-09-13T17:42:03.323065+0000","last_peered":"2024-09-13T17:42:03.323065+0000","last_clean":"2024-09-13T17:42:03.323065+0000","last_became_active":"2024-09-13T17:42:02.358808+0000","last_became_peered":"2024-09-13T17:42:02.358808+0000","last_unstale":"2024-09-13T17:42:03.323065+0000","last_undegraded":"2024-09-13T17:42:03.323065+0000","last_fullsized":"2024-09-13T17:42:03.323065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:37.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913042+0000","last_change":"2024-09-13T17:42:01.987023+0000","last_active":"2024-09-13T17:42:03.913042+0000","last_peered":"2024-09-13T17:42:03.913042+0000","last_clean":"2024-09-13T17:42:03.913042+0000","last_became_active":"2024-09-13T17:42:01.986220+0000","last_became_peered":"2024-09-13T17:42:01.986220+0000","last_unstale":"2024-09-13T17:42:03.913042+0000","last_undegraded":"2024-09-13T17:42:03.913042+0000","last_fullsized":"2024-09-13T17:42:03.913042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:12.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912685+0000","last_change":"2024-09-13T17:42:02.338763+0000","last_active":"2024-09-13T17:42:03.912685+0000","last_peered":"2024-09-13T17:42:03.912685+0000","last_clean":"2024-09-13T17:42:03.912685+0000","last_became_active":"2024-09-13T17:42:02.338494+0000","last_became_peered":"2024-09-13T17:42:02.338494+0000","last_unstale":"2024-09-13T17:42:03.912685+0000","last_undegraded":"2024-09-13T17:42:03.912685+0000","last_fullsized":"2024-09-13T17:42:03.912685+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323462+0000","last_change":"2024-09-13T17:42:01.986108+0000","last_active":"2024-09-13T17:42:03.323462+0000","last_peered":"2024-09-13T17:42:03.323462+0000","last_clean":"2024-09-13T17:42:03.323462+0000","last_became_active":"2024-09-13T17:42:01.985698+0000","last_became_peered":"2024-09-13T17:42:01.985698+0000","last_unstale":"2024-09-13T17:42:03.323462+0000","last_undegraded":"2024-09-13T17:42:03.323462+0000","last_fullsized":"2024-09-13T17:42:03.323462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":42001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":633029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":741376,"data_stored":631429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:10.003 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:42:10.003 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T17:42:10.004 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T17:42:10.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T17:42:10.004 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T17:42:10.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.165+0000 7f0652f0c640 1 Processor -- start 2024-09-13T17:42:10.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.165+0000 7f0652f0c640 1 -- start start 2024-09-13T17:42:10.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0652f0c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f064c134670 0x7f064c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0652f0c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f064c071670 con 0x7f064c06e5c0 2024-09-13T17:42:10.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0652f0c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f064c071360 con 0x7f064c134670 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0650ca8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f064c134670 0x7f064c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f064c06e5c0 0x7f064c06ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33708/0 (socket says 172.21.15.96:33708) 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064bfff640 1 -- 172.21.15.96:0/1148716454 learned_addr learned my addr 172.21.15.96:0/1148716454 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064b7fe640 1 -- 172.21.15.96:0/1148716454 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 692409237 0 0) 0x7f064c071670 con 0x7f064c06e5c0 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064b7fe640 1 -- 172.21.15.96:0/1148716454 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f063c003530 con 0x7f064c06e5c0 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0650ca8640 1 -- 172.21.15.96:0/1148716454 >> v1:172.21.15.96:6789/0 conn(0x7f064c06e5c0 legacy=0x7f064c06ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0650ca8640 1 -- 172.21.15.96:0/1148716454 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f064c0a7200 con 0x7f064c134670 2024-09-13T17:42:10.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f0650ca8640 1 --2- 172.21.15.96:0/1148716454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c134670 0x7f064c134a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0640009740 tx=0x7f064001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44d62d997f69518 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064b7fe640 1 -- 172.21.15.96:0/1148716454 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06400ab530 con 0x7f064c134670 2024-09-13T17:42:10.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064b7fe640 1 -- 172.21.15.96:0/1148716454 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06400a7050 con 0x7f064c134670 2024-09-13T17:42:10.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.169+0000 7f064b7fe640 1 -- 172.21.15.96:0/1148716454 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06400ab7c0 con 0x7f064c134670 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1148716454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c134670 msgr2=0x7f064c134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1148716454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c134670 0x7f064c134a50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0640009740 tx=0x7f064001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1148716454 shutdown_connections 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1148716454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c134670 0x7f064c134a50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1148716454 >> 172.21.15.96:0/1148716454 conn(0x7f064c098a60 msgr2=0x7f064c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1148716454 shutdown_connections 2024-09-13T17:42:10.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1148716454 wait complete. 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 Processor -- start 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- start start 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c0a6440 con 0x7f064c06e5c0 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0650ca8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0650ca8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50572/0 (socket says 172.21.15.96:50572) 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0650ca8640 1 -- 172.21.15.96:0/1507423339 learned_addr learned my addr 172.21.15.96:0/1507423339 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0650ca8640 1 -- 172.21.15.96:0/1507423339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640009420 con 0x7f064c06e5c0 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0650ca8640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0640019fc0 tx=0x7f0640019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0640003b60 con 0x7f064c06e5c0 2024-09-13T17:42:10.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0640003cc0 con 0x7f064c06e5c0 2024-09-13T17:42:10.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c09ebe0 con 0x7f064c06e5c0 2024-09-13T17:42:10.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06400b4400 con 0x7f064c06e5c0 2024-09-13T17:42:10.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c09fdf0 con 0x7f064c06e5c0 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c09d520 con 0x7f064c06e5c0 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f06400b2040 con 0x7f064c06e5c0 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 0x7f0630037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f06400e19e0 con 0x7f064c06e5c0 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.173+0000 7f064bfff640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 0x7f0630037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.177+0000 7f064bfff640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 0x7f0630037910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0638000f70 tx=0x7f06380010b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.181+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06400bc360 con 0x7f064c06e5c0 2024-09-13T17:42:10.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.381+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f064c06a1b0 con 0x7f064c06e5c0 2024-09-13T17:42:10.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.385+0000 7f06497fa640 1 -- 172.21.15.96:0/1507423339 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+83112 (secure 0 0 0) 0x7f06400b0030 con 0x7f064c06e5c0 2024-09-13T17:42:10.387 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 msgr2=0x7f0630037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 0x7f0630037910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0638000f70 tx=0x7f06380010b0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 msgr2=0x7f064c09d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0640019fc0 tx=0x7f0640019130 comp rx=0 tx=0).stop 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 shutdown_connections 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0630035450 0x7f0630037910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 --2- 172.21.15.96:0/1507423339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f064c06e5c0 0x7f064c09d920 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 >> 172.21.15.96:0/1507423339 conn(0x7f064c098a60 msgr2=0x7f064c12e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 shutdown_connections 2024-09-13T17:42:10.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.389+0000 7f0652f0c640 1 -- 172.21.15.96:0/1507423339 wait complete. 2024-09-13T17:42:10.404 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":762662892},{"type":"v1","addr":"172.21.15.96:6825","nonce":762662892}]},"active_addr":"172.21.15.96:6825/762662892","active_change":"2024-09-13T17:41:45.787639+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2609370247}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1797507976}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":3080912124}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":4173745824}]}]} 2024-09-13T17:42:10.406 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T17:42:10.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T17:42:10.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:10.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.525+0000 7f99c9c18640 1 Processor -- start 2024-09-13T17:42:10.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.525+0000 7f99c9c18640 1 -- start start 2024-09-13T17:42:10.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99c4134e90 0x7f99c4071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c4070de0 con 0x7f99c4135270 2024-09-13T17:42:10.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c4071d60 con 0x7f99c4134e90 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99c4134e90 0x7f99c4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f99c4134e90 0x7f99c4071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50580/0 (socket says 172.21.15.96:50580) 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c2ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f99c4135270 0x7f99c406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33716/0 (socket says 172.21.15.96:33716) 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 -- 172.21.15.96:0/4265851275 learned_addr learned my addr 172.21.15.96:0/4265851275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 -- 172.21.15.96:0/4265851275 >> v1:172.21.15.96:6789/0 conn(0x7f99c4135270 legacy=0x7f99c406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 -- 172.21.15.96:0/4265851275 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c406f110 con 0x7f99c4134e90 2024-09-13T17:42:10.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c27fc640 1 -- 172.21.15.96:0/4265851275 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2138883256 0 0) 0x7f99c4070de0 con 0x7f99c4135270 2024-09-13T17:42:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c37fe640 1 --2- 172.21.15.96:0/4265851275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c4071530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0097f0 tx=0x7f99ac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e7c8fe90dc05d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c27fc640 1 -- 172.21.15.96:0/4265851275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ac0ab530 con 0x7f99c4134e90 2024-09-13T17:42:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c27fc640 1 -- 172.21.15.96:0/4265851275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac0a7050 con 0x7f99c4134e90 2024-09-13T17:42:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c27fc640 1 -- 172.21.15.96:0/4265851275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ac0ab7e0 con 0x7f99c4134e90 2024-09-13T17:42:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- 172.21.15.96:0/4265851275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 msgr2=0x7f99c4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 --2- 172.21.15.96:0/4265851275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c4071530 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0097f0 tx=0x7f99ac01c450 comp rx=0 tx=0).stop 2024-09-13T17:42:10.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- 172.21.15.96:0/4265851275 shutdown_connections 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 --2- 172.21.15.96:0/4265851275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c4071530 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- 172.21.15.96:0/4265851275 >> 172.21.15.96:0/4265851275 conn(0x7f99c4098a60 msgr2=0x7f99c409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- 172.21.15.96:0/4265851275 shutdown_connections 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.529+0000 7f99c9c18640 1 -- 172.21.15.96:0/4265851275 wait complete. 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 Processor -- start 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 -- start start 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40a4e40 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50582/0 (socket says 172.21.15.96:50582) 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c37fe640 1 -- 172.21.15.96:0/32651258 learned_addr learned my addr 172.21.15.96:0/32651258 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c37fe640 1 -- 172.21.15.96:0/32651258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0094d0 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c37fe640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99ac01a080 tx=0x7f99ac01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ac0b6420 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac0b6580 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c40a4590 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99ac0b6840 con 0x7f99c4134e90 2024-09-13T17:42:10.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c413d530 con 0x7f99c4134e90 2024-09-13T17:42:10.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f99ac0b2040 con 0x7f99c4134e90 2024-09-13T17:42:10.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c40a48e0 con 0x7f99c4134e90 2024-09-13T17:42:10.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 0x7f99a4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f99ac0e1b70 con 0x7f99c4134e90 2024-09-13T17:42:10.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 0x7f99a4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.533+0000 7f99c2ffd640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 0x7f99a4037910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99b4007fd0 tx=0x7f99b4007940 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.537+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f99ac0b69a0 con 0x7f99c4134e90 2024-09-13T17:42:10.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.697+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c413d000 con 0x7f99c4134e90 2024-09-13T17:42:10.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.697+0000 7f99bbfff640 1 -- 172.21.15.96:0/32651258 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f99ac0b0030 con 0x7f99c4134e90 2024-09-13T17:42:10.699 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:10.699 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 msgr2=0x7f99a4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 0x7f99a4037910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f99b4007fd0 tx=0x7f99b4007940 comp rx=0 tx=0).stop 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 msgr2=0x7f99c40a2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f99ac01a080 tx=0x7f99ac01ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 shutdown_connections 2024-09-13T17:42:10.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f99a4035450 0x7f99a4037910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 --2- 172.21.15.96:0/32651258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99c4134e90 0x7f99c40a2990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 >> 172.21.15.96:0/32651258 conn(0x7f99c4098a60 msgr2=0x7f99c40996b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 shutdown_connections 2024-09-13T17:42:10.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.701+0000 7f99c9c18640 1 -- 172.21.15.96:0/32651258 wait complete. 2024-09-13T17:42:10.717 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T17:42:10.717 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:10.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991c9af640 1 Processor -- start 2024-09-13T17:42:10.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991c9af640 1 -- start start 2024-09-13T17:42:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991c9af640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f991809d690 0x7f9918138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991c9af640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9918138050 con 0x7f99181387d0 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991c9af640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991809c410 con 0x7f991809d690 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f991809d690 0x7f9918138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f9915d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f99181387d0 0x7f9918134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33718/0 (socket says 172.21.15.96:33718) 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f991809d690 0x7f9918138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50598/0 (socket says 172.21.15.96:50598) 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f9915d9b640 1 -- 172.21.15.96:0/3674374690 learned_addr learned my addr 172.21.15.96:0/3674374690 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991659c640 1 -- 172.21.15.96:0/3674374690 >> v1:172.21.15.96:6789/0 conn(0x7f99181387d0 legacy=0x7f9918134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991559a640 1 -- 172.21.15.96:0/3674374690 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432390613 0 0) 0x7f9900001120 con 0x7f99181387d0 2024-09-13T17:42:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.837+0000 7f991659c640 1 -- 172.21.15.96:0/3674374690 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991809cba0 con 0x7f991809d690 2024-09-13T17:42:10.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.841+0000 7f991659c640 1 --2- 172.21.15.96:0/3674374690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f9918138290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9908009740 tx=0x7f990801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a6b36f6403afb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.841+0000 7f991559a640 1 -- 172.21.15.96:0/3674374690 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99080a8840 con 0x7f991809d690 2024-09-13T17:42:10.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.841+0000 7f991559a640 1 -- 172.21.15.96:0/3674374690 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99080a89a0 con 0x7f991809d690 2024-09-13T17:42:10.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.841+0000 7f991559a640 1 -- 172.21.15.96:0/3674374690 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99080a8c60 con 0x7f991809d690 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3674374690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 msgr2=0x7f9918138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 --2- 172.21.15.96:0/3674374690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f9918138290 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9908009740 tx=0x7f990801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3674374690 shutdown_connections 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 --2- 172.21.15.96:0/3674374690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f9918138290 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3674374690 >> 172.21.15.96:0/3674374690 conn(0x7f991806bc60 msgr2=0x7f991806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3674374690 shutdown_connections 2024-09-13T17:42:10.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3674374690 wait complete. 2024-09-13T17:42:10.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 Processor -- start 2024-09-13T17:42:10.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- start start 2024-09-13T17:42:10.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180a4360 con 0x7f991809d690 2024-09-13T17:42:10.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50610/0 (socket says 172.21.15.96:50610) 2024-09-13T17:42:10.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991659c640 1 -- 172.21.15.96:0/3130779385 learned_addr learned my addr 172.21.15.96:0/3130779385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:10.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991659c640 1 -- 172.21.15.96:0/3130779385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9908009420 con 0x7f991809d690 2024-09-13T17:42:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991659c640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f990801c840 tx=0x7f990801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99080b0840 con 0x7f991809d690 2024-09-13T17:42:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99080b09a0 con 0x7f991809d690 2024-09-13T17:42:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99180a7040 con 0x7f991809d690 2024-09-13T17:42:10.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f99080b0c60 con 0x7f991809d690 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99180a5db0 con 0x7f991809d690 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991809bc00 con 0x7f991809d690 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f99080b7040 con 0x7f991809d690 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 0x7f98f8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.845+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f99080e5350 con 0x7f991809d690 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.849+0000 7f9915d9b640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 0x7f98f8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:10.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.849+0000 7f9915d9b640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 0x7f98f8037910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c10 tx=0x7f9900001260 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:10.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:10.853+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f991809bc00 con 0x7f991809d690 2024-09-13T17:42:11.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.013+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f991809bc00 con 0x7f991809d690 2024-09-13T17:42:11.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.013+0000 7f990effd640 1 -- 172.21.15.96:0/3130779385 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f991809bc00 con 0x7f991809d690 2024-09-13T17:42:11.014 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:11.014 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 msgr2=0x7f98f8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 0x7f98f8037910 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9900000c10 tx=0x7f9900001260 comp rx=0 tx=0).stop 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 msgr2=0x7f99180a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f990801c840 tx=0x7f990801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 shutdown_connections 2024-09-13T17:42:11.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98f8035450 0x7f98f8037910 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 --2- 172.21.15.96:0/3130779385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f991809d690 0x7f99180a6b00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 >> 172.21.15.96:0/3130779385 conn(0x7f991806bc60 msgr2=0x7f9918099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 shutdown_connections 2024-09-13T17:42:11.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.017+0000 7f991c9af640 1 -- 172.21.15.96:0/3130779385 wait complete. 2024-09-13T17:42:11.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T17:42:11.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T17:42:11.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T17:42:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebf8ef640 1 Processor -- start 2024-09-13T17:42:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebf8ef640 1 -- start start 2024-09-13T17:42:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbaddd89640 1 Processor -- start 2024-09-13T17:42:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbaddd89640 1 -- start start 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebf8ef640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2eb813af50 0x7f2eb813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebf8ef640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb8138050 con 0x7f2eb809d690 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebf8ef640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb809c410 con 0x7f2eb813af50 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebce8a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2eb813af50 0x7f2eb813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebd68b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2eb809d690 0x7f2eb8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33734/0 (socket says 172.21.15.96:33734) 2024-09-13T17:42:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebd68b640 1 -- 172.21.15.96:0/4240647472 learned_addr learned my addr 172.21.15.96:0/4240647472 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbaddd89640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbad809ba80 0x7fbad813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbaddd89640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad809a180 con 0x7fbad8137110 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbaddd89640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad8137c00 con 0x7fbad809ba80 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad77fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbad8137110 0x7fbad81374f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33736/0 (socket says 172.21.15.96:33736) 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad77fe640 1 -- 172.21.15.96:0/3829311007 learned_addr learned my addr 172.21.15.96:0/3829311007 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:11.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebce8a640 1 -- 172.21.15.96:0/4240647472 >> v1:172.21.15.96:6789/0 conn(0x7f2eb809d690 legacy=0x7f2eb8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7f2ebce8a640 1 -- 172.21.15.96:0/4240647472 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb809cba0 con 0x7f2eb813af50 2024-09-13T17:42:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad6ffd640 1 --2- 172.21.15.96:0/3829311007 >> v2:172.21.15.96:3300/0 conn(0x7fbad809ba80 0x7fbad813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396645259 0 0) 0x7fbad809a180 con 0x7fbad8137110 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbac0003530 con 0x7fbad8137110 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad6ffd640 1 -- 172.21.15.96:0/3829311007 >> v1:172.21.15.96:6789/0 conn(0x7fbad8137110 legacy=0x7fbad81374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1330928677 0 0) 0x7fbac0003530 con 0x7fbad8137110 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.141+0000 7fbad6ffd640 1 -- 172.21.15.96:0/3829311007 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad809adc0 con 0x7fbad809ba80 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebce8a640 1 --2- 172.21.15.96:0/4240647472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb813af50 0x7f2eb813d340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009740 tx=0x7f2ea801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab13b03886e889a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2eaffff640 1 -- 172.21.15.96:0/4240647472 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea80a8840 con 0x7f2eb813af50 2024-09-13T17:42:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2eaffff640 1 -- 172.21.15.96:0/4240647472 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea80a89a0 con 0x7f2eb813af50 2024-09-13T17:42:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2eaffff640 1 -- 172.21.15.96:0/4240647472 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ea80a8c60 con 0x7f2eb813af50 2024-09-13T17:42:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbad6ffd640 1 --2- 172.21.15.96:0/3829311007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad813d0f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009740 tx=0x7fbacc01c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc73f93d1213f6e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbac80015b0 con 0x7fbad809ba80 2024-09-13T17:42:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc0a7050 con 0x7fbad809ba80 2024-09-13T17:42:11.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbad67fc640 1 -- 172.21.15.96:0/3829311007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbacc0ab400 con 0x7fbad809ba80 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/4240647472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb813af50 msgr2=0x7f2eb813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/4240647472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb813af50 0x7f2eb813d340 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8009740 tx=0x7f2ea801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/4240647472 shutdown_connections 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/4240647472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb813af50 0x7f2eb813d340 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/4240647472 >> 172.21.15.96:0/4240647472 conn(0x7f2eb806bc60 msgr2=0x7f2eb806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/4240647472 shutdown_connections 2024-09-13T17:42:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/4240647472 wait complete. 2024-09-13T17:42:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- 172.21.15.96:0/3829311007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 msgr2=0x7fbad813d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 --2- 172.21.15.96:0/3829311007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad813d0f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbacc009740 tx=0x7fbacc01c2b0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- 172.21.15.96:0/3829311007 shutdown_connections 2024-09-13T17:42:11.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 --2- 172.21.15.96:0/3829311007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad813d0f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- 172.21.15.96:0/3829311007 >> 172.21.15.96:0/3829311007 conn(0x7fbad806e020 msgr2=0x7fbad8070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- 172.21.15.96:0/3829311007 shutdown_connections 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- 172.21.15.96:0/3829311007 wait complete. 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff919962640 1 Processor -- start 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff919962640 1 -- start start 2024-09-13T17:42:11.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff919962640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff9140a6510 0x7ff91413af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff919962640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff914070600 con 0x7ff9140a4900 2024-09-13T17:42:11.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff919962640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff914070830 con 0x7ff9140a6510 2024-09-13T17:42:11.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 Processor -- start 2024-09-13T17:42:11.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 Processor -- start 2024-09-13T17:42:11.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff9127fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff9140a6510 0x7ff91413af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff912ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff9140a4900 0x7ff9140a4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33748/0 (socket says 172.21.15.96:33748) 2024-09-13T17:42:11.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff9127fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff9140a6510 0x7ff91413af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50650/0 (socket says 172.21.15.96:50650) 2024-09-13T17:42:11.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7ff912ffd640 1 -- 172.21.15.96:0/1661719611 learned_addr learned my addr 172.21.15.96:0/1661719611 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:11.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7fbaddd89640 1 -- start start 2024-09-13T17:42:11.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.145+0000 7f2ebf8ef640 1 -- start start 2024-09-13T17:42:11.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7ff911ffb640 1 -- 172.21.15.96:0/1661719611 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3356701089 0 0) 0x7ff914070600 con 0x7ff9140a4900 2024-09-13T17:42:11.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7ff9127fc640 1 -- 172.21.15.96:0/1661719611 >> v1:172.21.15.96:6789/0 conn(0x7ff9140a4900 legacy=0x7ff9140a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7ff9127fc640 1 -- 172.21.15.96:0/1661719611 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140a5c70 con 0x7ff9140a6510 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad806eeb0 con 0x7fbad809ba80 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebf8ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebf8ef640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb80a2be0 con 0x7f2eb809d690 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50664/0 (socket says 172.21.15.96:50664) 2024-09-13T17:42:11.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad77fe640 1 -- 172.21.15.96:0/2425075757 learned_addr learned my addr 172.21.15.96:0/2425075757 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebd68b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebd68b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50678/0 (socket says 172.21.15.96:50678) 2024-09-13T17:42:11.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebd68b640 1 -- 172.21.15.96:0/3231599253 learned_addr learned my addr 172.21.15.96:0/3231599253 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad77fe640 1 -- 172.21.15.96:0/2425075757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009420 con 0x7fbad809ba80 2024-09-13T17:42:11.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebd68b640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea8009420 con 0x7f2eb809d690 2024-09-13T17:42:11.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad77fe640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c6d0 tx=0x7fbac800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebd68b640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c380 tx=0x7f2eb400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbac8005a10 con 0x7fbad809ba80 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac8005b70 con 0x7fbad809ba80 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbac8005e50 con 0x7fbad809ba80 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad811f670 con 0x7fbad809ba80 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eb4005c20 con 0x7f2eb809d690 2024-09-13T17:42:11.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4005d80 con 0x7f2eb809d690 2024-09-13T17:42:11.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb80a51a0 con 0x7f2eb809d690 2024-09-13T17:42:11.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2eb4007200 con 0x7f2eb809d690 2024-09-13T17:42:11.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb80a35d0 con 0x7f2eb809d690 2024-09-13T17:42:11.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad811e750 con 0x7fbad809ba80 2024-09-13T17:42:11.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2eb809c100 con 0x7f2eb809d690 2024-09-13T17:42:11.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fbac8007070 con 0x7fbad809ba80 2024-09-13T17:42:11.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 0x7fbac4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fbac8046d10 con 0x7fbad809ba80 2024-09-13T17:42:11.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2eb4007360 con 0x7f2eb809d690 2024-09-13T17:42:11.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad6ffd640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 0x7fbac4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 0x7f2e9c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 0x7fbad8136070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 --> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbad811e3e0 con 0x7fbad813fe50 2024-09-13T17:42:11.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f2eb4047520 con 0x7f2eb809d690 2024-09-13T17:42:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebce8a640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 0x7f2e9c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 0x7f2e9c03e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2e9c03ee20 con 0x7f2e9c03c310 2024-09-13T17:42:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2eb4047920 con 0x7f2eb809d690 2024-09-13T17:42:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad7fff640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 0x7fbad8136070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebde8c640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 0x7f2e9c03e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebce8a640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 0x7f2e9c037910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b2000 tx=0x7f2ea80aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7fbad7fff640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 0x7fbad8136070 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.149+0000 7f2ebde8c640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 0x7f2e9c03e750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7fbad6ffd640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 0x7fbac4037910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0ad000 tx=0x7fbacc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff9127fc640 1 --2- 172.21.15.96:0/1661719611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a6510 0x7ff91413af50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff908009740 tx=0x7ff90801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c52a8483c0d44e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff911ffb640 1 -- 172.21.15.96:0/1661719611 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9080ab530 con 0x7ff9140a6510 2024-09-13T17:42:11.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff911ffb640 1 -- 172.21.15.96:0/1661719611 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9080a7050 con 0x7ff9140a6510 2024-09-13T17:42:11.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff911ffb640 1 -- 172.21.15.96:0/1661719611 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9080ab7e0 con 0x7ff9140a6510 2024-09-13T17:42:11.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== osd.1 v2:172.21.15.96:6816/3848754233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f2e9c03ee20 con 0x7f2e9c03c310 2024-09-13T17:42:11.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- 172.21.15.96:0/1661719611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a6510 msgr2=0x7ff91413af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 --2- 172.21.15.96:0/1661719611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a6510 0x7ff91413af50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff908009740 tx=0x7ff90801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:11.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== osd.0 v2:172.21.15.96:6808/749556932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbad811e3e0 con 0x7fbad813fe50 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- 172.21.15.96:0/1661719611 shutdown_connections 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 --2- 172.21.15.96:0/1661719611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a6510 0x7ff91413af50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- 172.21.15.96:0/1661719611 >> 172.21.15.96:0/1661719611 conn(0x7ff91406a150 msgr2=0x7ff91406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- 172.21.15.96:0/1661719611 shutdown_connections 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- 172.21.15.96:0/1661719611 wait complete. 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 Processor -- start 2024-09-13T17:42:11.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- start start 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff919962640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9141c9e40 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50690/0 (socket says 172.21.15.96:50690) 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.153+0000 7ff912ffd640 1 -- 172.21.15.96:0/1671221234 learned_addr learned my addr 172.21.15.96:0/1671221234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.157+0000 7ff912ffd640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff908009420 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.157+0000 7ff912ffd640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c420 tx=0x7ff90400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.157+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9040074d0 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.157+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff904007630 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.157+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff904004960 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9141368a0 con 0x7ff9140a4900 2024-09-13T17:42:11.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91409bb00 con 0x7ff9140a4900 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff91413dcb0 con 0x7ff9140a4900 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff904004ac0 con 0x7ff9140a4900 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 0x7ff900037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff904047110 con 0x7ff9140a4900 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 0x7ff90003e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff90003ee20 con 0x7ff90003c310 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff9127fc640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 0x7ff900037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff904047510 con 0x7ff9140a4900 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff9137fe640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 0x7ff90003e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff9127fc640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 0x7ff900037910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ad000 tx=0x7ff9080a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff9137fe640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 0x7ff90003e750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.161+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== osd.2 v2:172.21.15.96:6800/654054578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7ff90003ee20 con 0x7ff90003c310 2024-09-13T17:42:11.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.173+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 --> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9141340b0 con 0x7ff90003c310 2024-09-13T17:42:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.173+0000 7ff8fbfff640 1 -- 172.21.15.96:0/1671221234 <== osd.2 v2:172.21.15.96:6800/654054578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9141340b0 con 0x7ff90003c310 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 msgr2=0x7ff90003e750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 0x7ff90003e750 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 msgr2=0x7ff900037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 0x7ff900037910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9080ad000 tx=0x7ff9080a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 msgr2=0x7ff91409bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff90400c420 tx=0x7ff90400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 shutdown_connections 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7ff90003c310 0x7ff90003e750 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff900035450 0x7ff900037910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 --2- 172.21.15.96:0/1671221234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff9140a4900 0x7ff91409bf90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 >> 172.21.15.96:0/1671221234 conn(0x7ff91406a150 msgr2=0x7ff914132190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 shutdown_connections 2024-09-13T17:42:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7ff919962640 1 -- 172.21.15.96:0/1671221234 wait complete. 2024-09-13T17:42:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.177+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 --> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2eb809b260 con 0x7f2e9c03c310 2024-09-13T17:42:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2eadffb640 1 -- 172.21.15.96:0/3231599253 <== osd.1 v2:172.21.15.96:6816/3848754233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2eb0011040 con 0x7f2e9c03c310 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 msgr2=0x7f2e9c03e750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 0x7f2e9c03e750 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 msgr2=0x7f2e9c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 0x7f2e9c037910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2ea80b2000 tx=0x7f2ea80aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 msgr2=0x7f2eb81c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400c380 tx=0x7f2eb400c840 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 shutdown_connections 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2e9c035450 0x7f2e9c037910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f2e9c03c310 0x7f2e9c03e750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 --2- 172.21.15.96:0/3231599253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2eb809d690 0x7f2eb81c94e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 >> 172.21.15.96:0/3231599253 conn(0x7f2eb806bc60 msgr2=0x7f2eb8131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 shutdown_connections 2024-09-13T17:42:11.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7f2ebf8ef640 1 -- 172.21.15.96:0/3231599253 wait complete. 2024-09-13T17:42:11.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 --> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbad8135c20 con 0x7fbad813fe50 2024-09-13T17:42:11.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbabffff640 1 -- 172.21.15.96:0/2425075757 <== osd.0 v2:172.21.15.96:6808/749556932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbad8135c20 con 0x7fbad813fe50 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 msgr2=0x7fbad8136070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 0x7fbad8136070 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 msgr2=0x7fbac4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 0x7fbac4037910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0ad000 tx=0x7fbacc008040 comp rx=0 tx=0).stop 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 msgr2=0x7fbad811f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbac800c6d0 tx=0x7fbac800cb90 comp rx=0 tx=0).stop 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 shutdown_connections 2024-09-13T17:42:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fbad813fe50 0x7fbad8136070 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbac4035450 0x7fbac4037910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 --2- 172.21.15.96:0/2425075757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbad809ba80 0x7fbad811f130 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 >> 172.21.15.96:0/2425075757 conn(0x7fbad806e020 msgr2=0x7fbad806f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 shutdown_connections 2024-09-13T17:42:11.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.181+0000 7fbaddd89640 1 -- 172.21.15.96:0/2425075757 wait complete. 2024-09-13T17:42:11.193 INFO:teuthology.orchestra.run.smithi096.stdout:34359738375 2024-09-13T17:42:11.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:42:11.199 INFO:teuthology.orchestra.run.smithi096.stdout:34359738376 2024-09-13T17:42:11.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:42:11.200 INFO:teuthology.orchestra.run.smithi096.stdout:34359738375 2024-09-13T17:42:11.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:42:11.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f3925c9f640 1 Processor -- start 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f3925c9f640 1 -- start start 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f3925c9f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f392013af50 0x7f392013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f3925c9f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3920138050 con 0x7f392009d690 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f3925c9f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f392009c410 con 0x7f392013af50 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f392013af50 0x7f392013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f392013af50 0x7f392013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50706/0 (socket says 172.21.15.96:50706) 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 -- 172.21.15.96:0/4246005159 learned_addr learned my addr 172.21.15.96:0/4246005159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 -- 172.21.15.96:0/4246005159 >> v1:172.21.15.96:6789/0 conn(0x7f392009d690 legacy=0x7f3920138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391e7fc640 1 -- 172.21.15.96:0/4246005159 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179412669 0 0) 0x7f3910001120 con 0x7f392009d690 2024-09-13T17:42:11.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 -- 172.21.15.96:0/4246005159 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392009cba0 con 0x7f392013af50 2024-09-13T17:42:11.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391effd640 1 --2- 172.21.15.96:0/4246005159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392013af50 0x7f392013d340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39140097f0 tx=0x7f391401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d5e3d71b6b3518 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391e7fc640 1 -- 172.21.15.96:0/4246005159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39140a8900 con 0x7f392013af50 2024-09-13T17:42:11.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391e7fc640 1 -- 172.21.15.96:0/4246005159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39140a8a60 con 0x7f392013af50 2024-09-13T17:42:11.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.301+0000 7f391e7fc640 1 -- 172.21.15.96:0/4246005159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39140a8d20 con 0x7f392013af50 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4246005159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392013af50 msgr2=0x7f392013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4246005159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392013af50 0x7f392013d340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f39140097f0 tx=0x7f391401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4246005159 shutdown_connections 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4246005159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392013af50 0x7f392013d340 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4246005159 >> 172.21.15.96:0/4246005159 conn(0x7f392006bc60 msgr2=0x7f392006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4246005159 shutdown_connections 2024-09-13T17:42:11.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4246005159 wait complete. 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 Processor -- start 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- start start 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39200a6f20 con 0x7f392009d690 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50722/0 (socket says 172.21.15.96:50722) 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f391f7fe640 1 -- 172.21.15.96:0/4013384225 learned_addr learned my addr 172.21.15.96:0/4013384225 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f391f7fe640 1 -- 172.21.15.96:0/4013384225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39140094d0 con 0x7f392009d690 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f391f7fe640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391000c390 tx=0x7f391000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3910005410 con 0x7f392009d690 2024-09-13T17:42:11.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910007030 con 0x7f392009d690 2024-09-13T17:42:11.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3910007310 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39200a4cb0 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39200a4ee0 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.305+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39200a4430 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.309+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f3910007540 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.309+0000 7f3907fff640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 0x7f390c0378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.309+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f3910046ef0 con 0x7f392009d690 2024-09-13T17:42:11.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.309+0000 7f391effd640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 0x7f390c0378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.309+0000 7f391effd640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 0x7f390c0378c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39140b2000 tx=0x7f39140aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.317+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f39200a4430 con 0x7f392009d690 2024-09-13T17:42:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f9d0a640 1 Processor -- start 2024-09-13T17:42:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f9d0a640 1 -- start start 2024-09-13T17:42:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff6b0063640 1 Processor -- start 2024-09-13T17:42:11.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff6b0063640 1 -- start start 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f9d0a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff7f409d690 0x7ff7f4138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f9d0a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f4138050 con 0x7ff7f41387d0 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f9d0a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f409c410 con 0x7ff7f409d690 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7f37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff7f409d690 0x7ff7f4138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.349+0000 7ff7ebfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff7f41387d0 0x7ff7f4134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33774/0 (socket says 172.21.15.96:33774) 2024-09-13T17:42:11.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7ebfff640 1 -- 172.21.15.96:0/995595663 learned_addr learned my addr 172.21.15.96:0/995595663 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff6a806ac90 0x7ff6a809e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6a809f0c0 con 0x7ff6a809ac70 2024-09-13T17:42:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6a809cde0 con 0x7ff6a806ac90 2024-09-13T17:42:11.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f37fe640 1 -- 172.21.15.96:0/995595663 >> v1:172.21.15.96:6789/0 conn(0x7ff7f41387d0 legacy=0x7ff7f4134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6ad5fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff6a806ac90 0x7ff6a809e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6addff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff6a809ac70 0x7ff6a809b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33776/0 (socket says 172.21.15.96:33776) 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6ad5fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff6a806ac90 0x7ff6a809e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50736/0 (socket says 172.21.15.96:50736) 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6addff640 1 -- 172.21.15.96:0/809853482 learned_addr learned my addr 172.21.15.96:0/809853482 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f2ffd640 1 -- 172.21.15.96:0/995595663 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145755658 0 0) 0x7ff7dc001120 con 0x7ff7f41387d0 2024-09-13T17:42:11.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f37fe640 1 -- 172.21.15.96:0/995595663 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f409cba0 con 0x7ff7f409d690 2024-09-13T17:42:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6ad5fe640 1 -- 172.21.15.96:0/809853482 >> v1:172.21.15.96:6789/0 conn(0x7ff6a809ac70 legacy=0x7ff6a809b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6ad5fe640 1 -- 172.21.15.96:0/809853482 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a809be10 con 0x7ff6a806ac90 2024-09-13T17:42:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f37fe640 1 --2- 172.21.15.96:0/995595663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f4138290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009740 tx=0x7ff7e401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46fad5eae033545 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f2ffd640 1 -- 172.21.15.96:0/995595663 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7e40a8840 con 0x7ff7f409d690 2024-09-13T17:42:11.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f2ffd640 1 -- 172.21.15.96:0/995595663 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e40a89a0 con 0x7ff7f409d690 2024-09-13T17:42:11.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f2ffd640 1 -- 172.21.15.96:0/995595663 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7e40a8c60 con 0x7ff7f409d690 2024-09-13T17:42:11.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6ad5fe640 1 --2- 172.21.15.96:0/809853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a809e970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff698009740 tx=0x7ff69801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb6e7236528670e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6acdfd640 1 -- 172.21.15.96:0/809853482 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6980ab530 con 0x7ff6a806ac90 2024-09-13T17:42:11.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6acdfd640 1 -- 172.21.15.96:0/809853482 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6980a7050 con 0x7ff6a806ac90 2024-09-13T17:42:11.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6acdfd640 1 -- 172.21.15.96:0/809853482 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6980ab7e0 con 0x7ff6a806ac90 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/995595663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 msgr2=0x7ff7f4138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/995595663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f4138290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4009740 tx=0x7ff7e401ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/995595663 shutdown_connections 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/995595663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f4138290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/995595663 >> 172.21.15.96:0/995595663 conn(0x7ff7f406bc60 msgr2=0x7ff7f406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/995595663 shutdown_connections 2024-09-13T17:42:11.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/995595663 wait complete. 2024-09-13T17:42:11.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- 172.21.15.96:0/809853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 msgr2=0x7ff6a809e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 --2- 172.21.15.96:0/809853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a809e970 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff698009740 tx=0x7ff69801c390 comp rx=0 tx=0).stop 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- 172.21.15.96:0/809853482 shutdown_connections 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 --2- 172.21.15.96:0/809853482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a809e970 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- 172.21.15.96:0/809853482 >> 172.21.15.96:0/809853482 conn(0x7ff6a806e350 msgr2=0x7ff6a8070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- 172.21.15.96:0/809853482 shutdown_connections 2024-09-13T17:42:11.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.353+0000 7ff6b0063640 1 -- 172.21.15.96:0/809853482 wait complete. 2024-09-13T17:42:11.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 Processor -- start 2024-09-13T17:42:11.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 -- start start 2024-09-13T17:42:11.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 Processor -- start 2024-09-13T17:42:11.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 -- start start 2024-09-13T17:42:11.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40a2840 con 0x7ff7f409d690 2024-09-13T17:42:11.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50738/0 (socket says 172.21.15.96:50738) 2024-09-13T17:42:11.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f37fe640 1 -- 172.21.15.96:0/920736449 learned_addr learned my addr 172.21.15.96:0/920736449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a809a3d0 con 0x7ff6a806ac90 2024-09-13T17:42:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6addff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6addff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50752/0 (socket says 172.21.15.96:50752) 2024-09-13T17:42:11.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6addff640 1 -- 172.21.15.96:0/3621020140 learned_addr learned my addr 172.21.15.96:0/3621020140 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f37fe640 1 -- 172.21.15.96:0/920736449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e4009420 con 0x7ff7f409d690 2024-09-13T17:42:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f37fe640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7e401b820 tx=0x7ff7e401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6addff640 1 -- 172.21.15.96:0/3621020140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009420 con 0x7ff6a806ac90 2024-09-13T17:42:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7e4004150 con 0x7ff7f409d690 2024-09-13T17:42:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e40042b0 con 0x7ff7f409d690 2024-09-13T17:42:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f40a5e90 con 0x7ff7f409d690 2024-09-13T17:42:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7e40bc720 con 0x7ff7f409d690 2024-09-13T17:42:11.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f40a6150 con 0x7ff7f409d690 2024-09-13T17:42:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6addff640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400c3a0 tx=0x7ff6a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a4005c20 con 0x7ff6a806ac90 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6a4005d80 con 0x7ff6a806ac90 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6a811ea10 con 0x7ff6a806ac90 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a813a360 con 0x7ff6a806ac90 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f409bc00 con 0x7ff7f409d690 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff7e40bc880 con 0x7ff7f409d690 2024-09-13T17:42:11.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 0x7ff7d4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6a4007200 con 0x7ff6a806ac90 2024-09-13T17:42:11.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6a80999b0 con 0x7ff6a806ac90 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.357+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff7e40e5a00 con 0x7ff7f409d690 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.361+0000 7ff7ebfff640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 0x7ff7d4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.361+0000 7ff7ebfff640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 0x7ff7d4037910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000c10 tx=0x7ff7dc001260 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7f409bc00 con 0x7ff7f409d690 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff6a4007360 con 0x7ff6a806ac90 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff69e7fc640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 0x7ff68c0378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff6a4047450 con 0x7ff6a806ac90 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff6ad5fe640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 0x7ff68c0378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.365+0000 7ff6ad5fe640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 0x7ff68c0378c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff698009740 tx=0x7ff6980a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:11.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.369+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6a4012400 con 0x7ff6a806ac90 2024-09-13T17:42:11.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.473+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3920138050 con 0x7f392009d690 2024-09-13T17:42:11.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.473+0000 7f3907fff640 1 -- 172.21.15.96:0/4013384225 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3920138050 con 0x7f392009d690 2024-09-13T17:42:11.475 INFO:teuthology.orchestra.run.smithi096.stdout:34359738373 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 msgr2=0x7f390c0378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 0x7f390c0378c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f39140b2000 tx=0x7f39140aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 msgr2=0x7f39200a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f391000c390 tx=0x7f391000c850 comp rx=0 tx=0).stop 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 shutdown_connections 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f390c035400 0x7f390c0378c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 --2- 172.21.15.96:0/4013384225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f392009d690 0x7f39200a69e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 >> 172.21.15.96:0/4013384225 conn(0x7f392006bc60 msgr2=0x7f3920139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 shutdown_connections 2024-09-13T17:42:11.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.477+0000 7f3925c9f640 1 -- 172.21.15.96:0/4013384225 wait complete. 2024-09-13T17:42:11.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738373 for osd.2 2024-09-13T17:42:11.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff7f409bc00 con 0x7ff7f409d690 2024-09-13T17:42:11.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f0ff9640 1 -- 172.21.15.96:0/920736449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7f409bc00 con 0x7ff7f409d690 2024-09-13T17:42:11.517 INFO:teuthology.orchestra.run.smithi096.stdout:34359738373 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 msgr2=0x7ff7d4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 0x7ff7d4037910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc000c10 tx=0x7ff7dc001260 comp rx=0 tx=0).stop 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 msgr2=0x7ff7f40a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff7e401b820 tx=0x7ff7e401d450 comp rx=0 tx=0).stop 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 shutdown_connections 2024-09-13T17:42:11.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff7d4035450 0x7ff7d4037910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 --2- 172.21.15.96:0/920736449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7f409d690 0x7ff7f40a68d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 >> 172.21.15.96:0/920736449 conn(0x7ff7f406bc60 msgr2=0x7ff7f409e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 shutdown_connections 2024-09-13T17:42:11.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.517+0000 7ff7f9d0a640 1 -- 172.21.15.96:0/920736449 wait complete. 2024-09-13T17:42:11.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff6a809e970 con 0x7ff6a806ac90 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff69e7fc640 1 -- 172.21.15.96:0/3621020140 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff6a4017020 con 0x7ff6a806ac90 2024-09-13T17:42:11.524 INFO:teuthology.orchestra.run.smithi096.stdout:34359738374 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 msgr2=0x7ff68c0378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 0x7ff68c0378c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff698009740 tx=0x7ff6980a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 msgr2=0x7ff6a80a5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400c3a0 tx=0x7ff6a400c860 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 shutdown_connections 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff68c035400 0x7ff68c0378c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 --2- 172.21.15.96:0/3621020140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff6a806ac90 0x7ff6a80a5fe0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:11.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 >> 172.21.15.96:0/3621020140 conn(0x7ff6a806e350 msgr2=0x7ff6a8137c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 shutdown_connections 2024-09-13T17:42:11.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:11.525+0000 7ff6b0063640 1 -- 172.21.15.96:0/3621020140 wait complete. 2024-09-13T17:42:11.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738373 for osd.0 2024-09-13T17:42:11.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738374 for osd.1 2024-09-13T17:42:12.493 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T17:42:12.536 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T17:42:12.542 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T17:42:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733fe29640 1 Processor -- start 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733fe29640 1 -- start start 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733fe29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f733809d690 0x7f7338138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733fe29640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7338138050 con 0x7f73381387d0 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733fe29640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733809c410 con 0x7f733809d690 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733dbc5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f733809d690 0x7f7338138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733dbc5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f733809d690 0x7f7338138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39334/0 (socket says 172.21.15.96:39334) 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733dbc5640 1 -- 172.21.15.96:0/1388584686 learned_addr learned my addr 172.21.15.96:0/1388584686 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733d3c4640 1 --1- 172.21.15.96:0/1388584686 >> v1:172.21.15.96:6789/0 conn(0x7f73381387d0 0x7f7338134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58892/0 (socket says 172.21.15.96:58892) 2024-09-13T17:42:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733dbc5640 1 -- 172.21.15.96:0/1388584686 >> v1:172.21.15.96:6789/0 conn(0x7f73381387d0 legacy=0x7f7338134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733dbc5640 1 -- 172.21.15.96:0/1388584686 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733809cba0 con 0x7f733809d690 2024-09-13T17:42:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.621+0000 7f733cbc3640 1 -- 172.21.15.96:0/1388584686 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2192695465 0 0) 0x7f7328001120 con 0x7f73381387d0 2024-09-13T17:42:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733dbc5640 1 --2- 172.21.15.96:0/1388584686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f7338138290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73340097f0 tx=0x7f733401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc64dffc93f4d7e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733cbc3640 1 -- 172.21.15.96:0/1388584686 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73340a8900 con 0x7f733809d690 2024-09-13T17:42:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733cbc3640 1 -- 172.21.15.96:0/1388584686 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73340a8a60 con 0x7f733809d690 2024-09-13T17:42:12.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733cbc3640 1 -- 172.21.15.96:0/1388584686 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73340a8d20 con 0x7f733809d690 2024-09-13T17:42:12.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- 172.21.15.96:0/1388584686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 msgr2=0x7f7338138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 --2- 172.21.15.96:0/1388584686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f7338138290 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73340097f0 tx=0x7f733401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- 172.21.15.96:0/1388584686 shutdown_connections 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 --2- 172.21.15.96:0/1388584686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f7338138290 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- 172.21.15.96:0/1388584686 >> 172.21.15.96:0/1388584686 conn(0x7f733806bc60 msgr2=0x7f733806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- 172.21.15.96:0/1388584686 shutdown_connections 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- 172.21.15.96:0/1388584686 wait complete. 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 Processor -- start 2024-09-13T17:42:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.625+0000 7f733fe29640 1 -- start start 2024-09-13T17:42:12.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733fe29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733fe29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73380a5c50 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733dbc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39342/0 (socket says 172.21.15.96:39342) 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733dbc5640 1 -- 172.21.15.96:0/557639543 learned_addr learned my addr 172.21.15.96:0/557639543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733dbc5640 1 -- 172.21.15.96:0/557639543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73340094d0 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733dbc5640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f733401c900 tx=0x7f733401ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73340bf070 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73340b09b0 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733fe29640 1 -- 172.21.15.96:0/557639543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73380a6d80 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73340b0c70 con 0x7f733809d690 2024-09-13T17:42:12.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733fe29640 1 -- 172.21.15.96:0/557639543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733809f500 con 0x7f733809d690 2024-09-13T17:42:12.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f73340b7070 con 0x7f733809d690 2024-09-13T17:42:12.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7314005260 con 0x7f733809d690 2024-09-13T17:42:12.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 0x7f731c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f73340e5120 con 0x7f733809d690 2024-09-13T17:42:12.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733d3c4640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 0x7f731c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.629+0000 7f733d3c4640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 0x7f731c037910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c10 tx=0x7f7328001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.633+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7334116db0 con 0x7f733809d690 2024-09-13T17:42:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5ebdd28640 1 Processor -- start 2024-09-13T17:42:12.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5ebdd28640 1 -- start start 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5ebdd28640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5eb813af50 0x7f5eb813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5ebdd28640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb8138050 con 0x7f5eb809d690 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5ebdd28640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb809c410 con 0x7f5eb813af50 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5eb813af50 0x7f5eb813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb77fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5eb809d690 0x7f5eb8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58908/0 (socket says 172.21.15.96:58908) 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb77fe640 1 -- 172.21.15.96:0/1671943237 learned_addr learned my addr 172.21.15.96:0/1671943237 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5eb813af50 0x7f5eb813d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39344/0 (socket says 172.21.15.96:39344) 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/1671943237 >> v1:172.21.15.96:6789/0 conn(0x7f5eb809d690 legacy=0x7f5eb8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb67fc640 1 -- 172.21.15.96:0/1671943237 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15496876 0 0) 0x7f5ea8001120 con 0x7f5eb809d690 2024-09-13T17:42:12.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.661+0000 7f5eb6ffd640 1 -- 172.21.15.96:0/1671943237 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb809cba0 con 0x7f5eb813af50 2024-09-13T17:42:12.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5eb6ffd640 1 --2- 172.21.15.96:0/1671943237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb813af50 0x7f5eb813d340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009740 tx=0x7f5eac01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f8b816ec1c1382 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5eb67fc640 1 -- 172.21.15.96:0/1671943237 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5eac0a8840 con 0x7f5eb813af50 2024-09-13T17:42:12.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5eb67fc640 1 -- 172.21.15.96:0/1671943237 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac0a89a0 con 0x7f5eb813af50 2024-09-13T17:42:12.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5eb67fc640 1 -- 172.21.15.96:0/1671943237 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5eac0a8c60 con 0x7f5eb813af50 2024-09-13T17:42:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 -- 172.21.15.96:0/1671943237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb813af50 msgr2=0x7f5eb813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/1671943237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb813af50 0x7f5eb813d340 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009740 tx=0x7f5eac01ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 -- 172.21.15.96:0/1671943237 shutdown_connections 2024-09-13T17:42:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/1671943237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb813af50 0x7f5eb813d340 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 -- 172.21.15.96:0/1671943237 >> 172.21.15.96:0/1671943237 conn(0x7f5eb806bc60 msgr2=0x7f5eb806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 -- 172.21.15.96:0/1671943237 shutdown_connections 2024-09-13T17:42:12.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 -- 172.21.15.96:0/1671943237 wait complete. 2024-09-13T17:42:12.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.665+0000 7f5ebdd28640 1 Processor -- start 2024-09-13T17:42:12.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 -- start start 2024-09-13T17:42:12.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80a6f20 con 0x7f5eb809d690 2024-09-13T17:42:12.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39360/0 (socket says 172.21.15.96:39360) 2024-09-13T17:42:12.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb77fe640 1 -- 172.21.15.96:0/2090646747 learned_addr learned my addr 172.21.15.96:0/2090646747 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb77fe640 1 -- 172.21.15.96:0/2090646747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac009420 con 0x7f5eb809d690 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb77fe640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c390 tx=0x7f5ea800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea8007410 con 0x7f5eb809d690 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8007590 con 0x7f5eb809d690 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb80a4cb0 con 0x7f5eb809d690 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ea80047f0 con 0x7f5eb809d690 2024-09-13T17:42:12.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb80a4ee0 con 0x7f5eb809d690 2024-09-13T17:42:12.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80a4430 con 0x7f5eb809d690 2024-09-13T17:42:12.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5ea8004950 con 0x7f5eb809d690 2024-09-13T17:42:12.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 0x7f5ea4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5ea8047290 con 0x7f5eb809d690 2024-09-13T17:42:12.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb6ffd640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 0x7f5ea4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.669+0000 7f5eb6ffd640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 0x7f5ea4037910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0b2000 tx=0x7f5eac0aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.673+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5eb80a4430 con 0x7f5eb809d690 2024-09-13T17:42:12.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.693+0000 7f9566e09640 1 Processor -- start 2024-09-13T17:42:12.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.693+0000 7f9566e09640 1 -- start start 2024-09-13T17:42:12.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f956013af50 0x7f956013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9560138050 con 0x7f956009d690 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f956009c410 con 0x7f956013af50 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9564ba5640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f956009d690 0x7f9560138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58922/0 (socket says 172.21.15.96:58922) 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f956013af50 0x7f956013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9564ba5640 1 -- 172.21.15.96:0/1294539399 learned_addr learned my addr 172.21.15.96:0/1294539399 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955f7fe640 1 -- 172.21.15.96:0/1294539399 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769945372 0 0) 0x7f9554001120 con 0x7f956009d690 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955f7fe640 1 -- 172.21.15.96:0/1294539399 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9550003530 con 0x7f956009d690 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955ffff640 1 -- 172.21.15.96:0/1294539399 >> v1:172.21.15.96:6789/0 conn(0x7f956009d690 legacy=0x7f9560138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955ffff640 1 -- 172.21.15.96:0/1294539399 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956009cba0 con 0x7f956013af50 2024-09-13T17:42:12.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955ffff640 1 --2- 172.21.15.96:0/1294539399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956013af50 0x7f956013d340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f954c009740 tx=0x7f954c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999e4b816867810d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955f7fe640 1 -- 172.21.15.96:0/1294539399 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f954c0a8840 con 0x7f956013af50 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955f7fe640 1 -- 172.21.15.96:0/1294539399 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954c0a89a0 con 0x7f956013af50 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f955f7fe640 1 -- 172.21.15.96:0/1294539399 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f954c0a8c40 con 0x7f956013af50 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- 172.21.15.96:0/1294539399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956013af50 msgr2=0x7f956013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 --2- 172.21.15.96:0/1294539399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956013af50 0x7f956013d340 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f954c009740 tx=0x7f954c01ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- 172.21.15.96:0/1294539399 shutdown_connections 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 --2- 172.21.15.96:0/1294539399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956013af50 0x7f956013d340 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- 172.21.15.96:0/1294539399 >> 172.21.15.96:0/1294539399 conn(0x7f956006bc60 msgr2=0x7f956006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- 172.21.15.96:0/1294539399 shutdown_connections 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.697+0000 7f9566e09640 1 -- 172.21.15.96:0/1294539399 wait complete. 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 Processor -- start 2024-09-13T17:42:12.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 -- start start 2024-09-13T17:42:12.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95600a5ea0 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9564ba5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9564ba5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39374/0 (socket says 172.21.15.96:39374) 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9564ba5640 1 -- 172.21.15.96:0/2977104736 learned_addr learned my addr 172.21.15.96:0/2977104736 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9564ba5640 1 -- 172.21.15.96:0/2977104736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954c009420 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9564ba5640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f955400c380 tx=0x7f955400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9554005c20 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9554005d80 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9554007220 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95601373d0 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95600a4440 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95600a5ff0 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f9554007380 con 0x7f956009d690 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 0x7f9544037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:12.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f95540468a0 con 0x7f956009d690 2024-09-13T17:42:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955ffff640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 0x7f9544037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:12.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.701+0000 7f955ffff640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 0x7f9544037910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f954c0b2000 tx=0x7f954c0aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:12.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.705+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95540198b0 con 0x7f956009d690 2024-09-13T17:42:12.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.793+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7314005510 con 0x7f733809d690 2024-09-13T17:42:12.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.793+0000 7f732e7fc640 1 -- 172.21.15.96:0/557639543 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f73340b5070 con 0x7f733809d690 2024-09-13T17:42:12.794 INFO:teuthology.orchestra.run.smithi096.stdout:34359738375 2024-09-13T17:42:12.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 msgr2=0x7f731c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 0x7f731c037910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7328000c10 tx=0x7f7328001320 comp rx=0 tx=0).stop 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 msgr2=0x7f73381cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f733401c900 tx=0x7f733401ba70 comp rx=0 tx=0).stop 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 shutdown_connections 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f731c035450 0x7f731c037910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 --2- 172.21.15.96:0/557639543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f733809d690 0x7f73381cc690 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 >> 172.21.15.96:0/557639543 conn(0x7f733806bc60 msgr2=0x7f73380994f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 shutdown_connections 2024-09-13T17:42:12.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.797+0000 7f731bfff640 1 -- 172.21.15.96:0/557639543 wait complete. 2024-09-13T17:42:12.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.2 2024-09-13T17:42:12.811 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:12.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5eb8138050 con 0x7f5eb809d690 2024-09-13T17:42:12.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5e9ffff640 1 -- 172.21.15.96:0/2090646747 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5eb8138050 con 0x7f5eb809d690 2024-09-13T17:42:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:34359738375 2024-09-13T17:42:12.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 msgr2=0x7f5ea4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 0x7f5ea4037910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0b2000 tx=0x7f5eac0aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:12.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 msgr2=0x7f5eb80a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.831 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c390 tx=0x7f5ea800c850 comp rx=0 tx=0).stop 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 shutdown_connections 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5ea4035450 0x7f5ea4037910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 --2- 172.21.15.96:0/2090646747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5eb809d690 0x7f5eb80a69e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.829+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 >> 172.21.15.96:0/2090646747 conn(0x7f5eb806bc60 msgr2=0x7f5eb8139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.833+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 shutdown_connections 2024-09-13T17:42:12.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.833+0000 7f5ebdd28640 1 -- 172.21.15.96:0/2090646747 wait complete. 2024-09-13T17:42:12.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.0 2024-09-13T17:42:12.846 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:12.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9560138050 con 0x7f956009d690 2024-09-13T17:42:12.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f955d7fa640 1 -- 172.21.15.96:0/2977104736 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9560138050 con 0x7f956009d690 2024-09-13T17:42:12.860 INFO:teuthology.orchestra.run.smithi096.stdout:34359738376 2024-09-13T17:42:12.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 msgr2=0x7f9544037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 0x7f9544037910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f954c0b2000 tx=0x7f954c0aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 msgr2=0x7f95600a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f955400c380 tx=0x7f955400c840 comp rx=0 tx=0).stop 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 shutdown_connections 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9544035450 0x7f9544037910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 --2- 172.21.15.96:0/2977104736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f956009d690 0x7f95600a6900 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 >> 172.21.15.96:0/2977104736 conn(0x7f956006bc60 msgr2=0x7f9560131290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 shutdown_connections 2024-09-13T17:42:12.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:12.861+0000 7f9566e09640 1 -- 172.21.15.96:0/2977104736 wait complete. 2024-09-13T17:42:12.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738376 got 34359738376 for osd.1 2024-09-13T17:42:12.877 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:12.877 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:42:12.878 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:13.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.021+0000 7f74c8208640 1 Processor -- start 2024-09-13T17:42:13.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.021+0000 7f74c8208640 1 -- start start 2024-09-13T17:42:13.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f74c009d690 0x7f74c0138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c0138050 con 0x7f74c01387d0 2024-09-13T17:42:13.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c009c410 con 0x7f74c009d690 2024-09-13T17:42:13.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c5fa4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f74c009d690 0x7f74c0138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c57a3640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f74c01387d0 0x7f74c0134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58926/0 (socket says 172.21.15.96:58926) 2024-09-13T17:42:13.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c57a3640 1 -- 172.21.15.96:0/1502130243 learned_addr learned my addr 172.21.15.96:0/1502130243 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:13.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c5fa4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f74c009d690 0x7f74c0138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39386/0 (socket says 172.21.15.96:39386) 2024-09-13T17:42:13.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c5fa4640 1 -- 172.21.15.96:0/1502130243 >> v1:172.21.15.96:6789/0 conn(0x7f74c01387d0 legacy=0x7f74c0134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c5fa4640 1 -- 172.21.15.96:0/1502130243 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c009cba0 con 0x7f74c009d690 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c5fa4640 1 --2- 172.21.15.96:0/1502130243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c0138290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009740 tx=0x7f74b001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d3b9cb494832e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c4fa2640 1 -- 172.21.15.96:0/1502130243 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74b00a8840 con 0x7f74c009d690 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c4fa2640 1 -- 172.21.15.96:0/1502130243 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00a89a0 con 0x7f74c009d690 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c4fa2640 1 -- 172.21.15.96:0/1502130243 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74b00a8c60 con 0x7f74c009d690 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- 172.21.15.96:0/1502130243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 msgr2=0x7f74c0138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 --2- 172.21.15.96:0/1502130243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c0138290 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f74b0009740 tx=0x7f74b001ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- 172.21.15.96:0/1502130243 shutdown_connections 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 --2- 172.21.15.96:0/1502130243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c0138290 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- 172.21.15.96:0/1502130243 >> 172.21.15.96:0/1502130243 conn(0x7f74c006bc60 msgr2=0x7f74c006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- 172.21.15.96:0/1502130243 shutdown_connections 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.025+0000 7f74c8208640 1 -- 172.21.15.96:0/1502130243 wait complete. 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 Processor -- start 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 -- start start 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c00a5d10 con 0x7f74c009d690 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c5fa4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c5fa4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39394/0 (socket says 172.21.15.96:39394) 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c5fa4640 1 -- 172.21.15.96:0/4191203682 learned_addr learned my addr 172.21.15.96:0/4191203682 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c5fa4640 1 -- 172.21.15.96:0/4191203682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b0009420 con 0x7f74c009d690 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c5fa4640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f74b001c840 tx=0x7f74b001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74b00b0840 con 0x7f74c009d690 2024-09-13T17:42:13.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b00b09a0 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74b00b0c60 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c00a4470 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c009f490 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f74b00b7040 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c009bc00 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 0x7f74a4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f74b00e5350 con 0x7f74c009d690 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c57a3640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 0x7f74a4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.029+0000 7f74c57a3640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 0x7f74a4037910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005890 tx=0x7f74bc007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.033+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74c009bc00 con 0x7f74c009d690 2024-09-13T17:42:13.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.177+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74c009c100 con 0x7f74a4035450 2024-09-13T17:42:13.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.193+0000 7f74b67fc640 1 -- 172.21.15.96:0/4191203682 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247338 (secure 0 0 0) 0x7f74bc001c00 con 0x7f74a4035450 2024-09-13T17:42:13.195 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:13.197 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 msgr2=0x7f74a4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 0x7f74a4037910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f74bc005890 tx=0x7f74bc007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 msgr2=0x7f74c00a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f74b001c840 tx=0x7f74b001b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 shutdown_connections 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74a4035450 0x7f74a4037910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 --2- 172.21.15.96:0/4191203682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74c009d690 0x7f74c00a6a60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 >> 172.21.15.96:0/4191203682 conn(0x7f74c006bc60 msgr2=0x7f74c0099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 shutdown_connections 2024-09-13T17:42:13.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.197+0000 7f74c8208640 1 -- 172.21.15.96:0/4191203682 wait complete. 2024-09-13T17:42:13.212 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-13T17:42:11.798043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":182,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82668,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032852,"statfs":{"total":289910292480,"available":289825640448,"internally_reserved":0,"allocated":1646592,"data_stored":1306459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.482736"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322958+0000","last_change":"2024-09-13T17:42:02.360541+0000","last_active":"2024-09-13T17:42:03.322958+0000","last_peered":"2024-09-13T17:42:03.322958+0000","last_clean":"2024-09-13T17:42:03.322958+0000","last_became_active":"2024-09-13T17:42:02.359334+0000","last_became_peered":"2024-09-13T17:42:02.359334+0000","last_unstale":"2024-09-13T17:42:03.322958+0000","last_undegraded":"2024-09-13T17:42:03.322958+0000","last_fullsized":"2024-09-13T17:42:03.322958+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:42.629895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264089+0000","last_change":"2024-09-13T17:42:02.357141+0000","last_active":"2024-09-13T17:42:04.264089+0000","last_peered":"2024-09-13T17:42:04.264089+0000","last_clean":"2024-09-13T17:42:04.264089+0000","last_became_active":"2024-09-13T17:42:02.356712+0000","last_became_peered":"2024-09-13T17:42:02.356712+0000","last_unstale":"2024-09-13T17:42:04.264089+0000","last_undegraded":"2024-09-13T17:42:04.264089+0000","last_fullsized":"2024-09-13T17:42:04.264089+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322398+0000","last_change":"2024-09-13T17:42:02.353311+0000","last_active":"2024-09-13T17:42:03.322398+0000","last_peered":"2024-09-13T17:42:03.322398+0000","last_clean":"2024-09-13T17:42:03.322398+0000","last_became_active":"2024-09-13T17:42:02.351787+0000","last_became_peered":"2024-09-13T17:42:02.351787+0000","last_unstale":"2024-09-13T17:42:03.322398+0000","last_undegraded":"2024-09-13T17:42:03.322398+0000","last_fullsized":"2024-09-13T17:42:03.322398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:35.554439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912292+0000","last_change":"2024-09-13T17:42:02.364812+0000","last_active":"2024-09-13T17:42:03.912292+0000","last_peered":"2024-09-13T17:42:03.912292+0000","last_clean":"2024-09-13T17:42:03.912292+0000","last_became_active":"2024-09-13T17:42:02.364256+0000","last_became_peered":"2024-09-13T17:42:02.364256+0000","last_unstale":"2024-09-13T17:42:03.912292+0000","last_undegraded":"2024-09-13T17:42:03.912292+0000","last_fullsized":"2024-09-13T17:42:03.912292+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322703+0000","last_change":"2024-09-13T17:42:02.359338+0000","last_active":"2024-09-13T17:42:03.322703+0000","last_peered":"2024-09-13T17:42:03.322703+0000","last_clean":"2024-09-13T17:42:03.322703+0000","last_became_active":"2024-09-13T17:42:02.355220+0000","last_became_peered":"2024-09-13T17:42:02.355220+0000","last_unstale":"2024-09-13T17:42:03.322703+0000","last_undegraded":"2024-09-13T17:42:03.322703+0000","last_fullsized":"2024-09-13T17:42:03.322703+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:41.277636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911521+0000","last_change":"2024-09-13T17:42:02.364229+0000","last_active":"2024-09-13T17:42:03.911521+0000","last_peered":"2024-09-13T17:42:03.911521+0000","last_clean":"2024-09-13T17:42:03.911521+0000","last_became_active":"2024-09-13T17:42:02.363257+0000","last_became_peered":"2024-09-13T17:42:02.363257+0000","last_unstale":"2024-09-13T17:42:03.911521+0000","last_undegraded":"2024-09-13T17:42:03.911521+0000","last_fullsized":"2024-09-13T17:42:03.911521+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321735+0000","last_change":"2024-09-13T17:42:02.353689+0000","last_active":"2024-09-13T17:42:03.321735+0000","last_peered":"2024-09-13T17:42:03.321735+0000","last_clean":"2024-09-13T17:42:03.321735+0000","last_became_active":"2024-09-13T17:42:02.353466+0000","last_became_peered":"2024-09-13T17:42:02.353466+0000","last_unstale":"2024-09-13T17:42:03.321735+0000","last_undegraded":"2024-09-13T17:42:03.321735+0000","last_fullsized":"2024-09-13T17:42:03.321735+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:23.813864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911081+0000","last_change":"2024-09-13T17:42:02.351016+0000","last_active":"2024-09-13T17:42:03.911081+0000","last_peered":"2024-09-13T17:42:03.911081+0000","last_clean":"2024-09-13T17:42:03.911081+0000","last_became_active":"2024-09-13T17:42:02.349745+0000","last_became_peered":"2024-09-13T17:42:02.349745+0000","last_unstale":"2024-09-13T17:42:03.911081+0000","last_undegraded":"2024-09-13T17:42:03.911081+0000","last_fullsized":"2024-09-13T17:42:03.911081+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264317+0000","last_change":"2024-09-13T17:42:02.348417+0000","last_active":"2024-09-13T17:42:04.264317+0000","last_peered":"2024-09-13T17:42:04.264317+0000","last_clean":"2024-09-13T17:42:04.264317+0000","last_became_active":"2024-09-13T17:42:02.348073+0000","last_became_peered":"2024-09-13T17:42:02.348073+0000","last_unstale":"2024-09-13T17:42:04.264317+0000","last_undegraded":"2024-09-13T17:42:04.264317+0000","last_fullsized":"2024-09-13T17:42:04.264317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322820+0000","last_change":"2024-09-13T17:42:02.348661+0000","last_active":"2024-09-13T17:42:03.322820+0000","last_peered":"2024-09-13T17:42:03.322820+0000","last_clean":"2024-09-13T17:42:03.322820+0000","last_became_active":"2024-09-13T17:42:02.348395+0000","last_became_peered":"2024-09-13T17:42:02.348395+0000","last_unstale":"2024-09-13T17:42:03.322820+0000","last_undegraded":"2024-09-13T17:42:03.322820+0000","last_fullsized":"2024-09-13T17:42:03.322820+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:44.538462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322196+0000","last_change":"2024-09-13T17:42:02.355321+0000","last_active":"2024-09-13T17:42:03.322196+0000","last_peered":"2024-09-13T17:42:03.322196+0000","last_clean":"2024-09-13T17:42:03.322196+0000","last_became_active":"2024-09-13T17:42:02.355024+0000","last_became_peered":"2024-09-13T17:42:02.355024+0000","last_unstale":"2024-09-13T17:42:03.322196+0000","last_undegraded":"2024-09-13T17:42:03.322196+0000","last_fullsized":"2024-09-13T17:42:03.322196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:24.061971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321999+0000","last_change":"2024-09-13T17:42:02.360863+0000","last_active":"2024-09-13T17:42:03.321999+0000","last_peered":"2024-09-13T17:42:03.321999+0000","last_clean":"2024-09-13T17:42:03.321999+0000","last_became_active":"2024-09-13T17:42:02.359969+0000","last_became_peered":"2024-09-13T17:42:02.359969+0000","last_unstale":"2024-09-13T17:42:03.321999+0000","last_undegraded":"2024-09-13T17:42:03.321999+0000","last_fullsized":"2024-09-13T17:42:03.321999+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322818+0000","last_change":"2024-09-13T17:42:02.357373+0000","last_active":"2024-09-13T17:42:03.322818+0000","last_peered":"2024-09-13T17:42:03.322818+0000","last_clean":"2024-09-13T17:42:03.322818+0000","last_became_active":"2024-09-13T17:42:02.354117+0000","last_became_peered":"2024-09-13T17:42:02.354117+0000","last_unstale":"2024-09-13T17:42:03.322818+0000","last_undegraded":"2024-09-13T17:42:03.322818+0000","last_fullsized":"2024-09-13T17:42:03.322818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:28.692862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321544+0000","last_change":"2024-09-13T17:42:02.359487+0000","last_active":"2024-09-13T17:42:03.321544+0000","last_peered":"2024-09-13T17:42:03.321544+0000","last_clean":"2024-09-13T17:42:03.321544+0000","last_became_active":"2024-09-13T17:42:02.358942+0000","last_became_peered":"2024-09-13T17:42:02.358942+0000","last_unstale":"2024-09-13T17:42:03.321544+0000","last_undegraded":"2024-09-13T17:42:03.321544+0000","last_fullsized":"2024-09-13T17:42:03.321544+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:25.373355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263024+0000","last_change":"2024-09-13T17:42:02.349044+0000","last_active":"2024-09-13T17:42:04.263024+0000","last_peered":"2024-09-13T17:42:04.263024+0000","last_clean":"2024-09-13T17:42:04.263024+0000","last_became_active":"2024-09-13T17:42:02.345647+0000","last_became_peered":"2024-09-13T17:42:02.345647+0000","last_unstale":"2024-09-13T17:42:04.263024+0000","last_undegraded":"2024-09-13T17:42:04.263024+0000","last_fullsized":"2024-09-13T17:42:04.263024+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911223+0000","last_change":"2024-09-13T17:42:02.363617+0000","last_active":"2024-09-13T17:42:03.911223+0000","last_peered":"2024-09-13T17:42:03.911223+0000","last_clean":"2024-09-13T17:42:03.911223+0000","last_became_active":"2024-09-13T17:42:02.362957+0000","last_became_peered":"2024-09-13T17:42:02.362957+0000","last_unstale":"2024-09-13T17:42:03.911223+0000","last_undegraded":"2024-09-13T17:42:03.911223+0000","last_fullsized":"2024-09-13T17:42:03.911223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264682+0000","last_change":"2024-09-13T17:42:02.349237+0000","last_active":"2024-09-13T17:42:04.264682+0000","last_peered":"2024-09-13T17:42:04.264682+0000","last_clean":"2024-09-13T17:42:04.264682+0000","last_became_active":"2024-09-13T17:42:02.348179+0000","last_became_peered":"2024-09-13T17:42:02.348179+0000","last_unstale":"2024-09-13T17:42:04.264682+0000","last_undegraded":"2024-09-13T17:42:04.264682+0000","last_fullsized":"2024-09-13T17:42:04.264682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912431+0000","last_change":"2024-09-13T17:42:02.363954+0000","last_active":"2024-09-13T17:42:03.912431+0000","last_peered":"2024-09-13T17:42:03.912431+0000","last_clean":"2024-09-13T17:42:03.912431+0000","last_became_active":"2024-09-13T17:42:02.363490+0000","last_became_peered":"2024-09-13T17:42:02.363490+0000","last_unstale":"2024-09-13T17:42:03.912431+0000","last_undegraded":"2024-09-13T17:42:03.912431+0000","last_fullsized":"2024-09-13T17:42:03.912431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322498+0000","last_change":"2024-09-13T17:42:02.344432+0000","last_active":"2024-09-13T17:42:03.322498+0000","last_peered":"2024-09-13T17:42:03.322498+0000","last_clean":"2024-09-13T17:42:03.322498+0000","last_became_active":"2024-09-13T17:42:02.344155+0000","last_became_peered":"2024-09-13T17:42:02.344155+0000","last_unstale":"2024-09-13T17:42:03.322498+0000","last_undegraded":"2024-09-13T17:42:03.322498+0000","last_fullsized":"2024-09-13T17:42:03.322498+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:24.986220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321991+0000","last_change":"2024-09-13T17:42:02.360682+0000","last_active":"2024-09-13T17:42:03.321991+0000","last_peered":"2024-09-13T17:42:03.321991+0000","last_clean":"2024-09-13T17:42:03.321991+0000","last_became_active":"2024-09-13T17:42:02.356733+0000","last_became_peered":"2024-09-13T17:42:02.356733+0000","last_unstale":"2024-09-13T17:42:03.321991+0000","last_undegraded":"2024-09-13T17:42:03.321991+0000","last_fullsized":"2024-09-13T17:42:03.321991+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:43.147144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321984+0000","last_change":"2024-09-13T17:42:02.360599+0000","last_active":"2024-09-13T17:42:03.321984+0000","last_peered":"2024-09-13T17:42:03.321984+0000","last_clean":"2024-09-13T17:42:03.321984+0000","last_became_active":"2024-09-13T17:42:02.357562+0000","last_became_peered":"2024-09-13T17:42:02.357562+0000","last_unstale":"2024-09-13T17:42:03.321984+0000","last_undegraded":"2024-09-13T17:42:03.321984+0000","last_fullsized":"2024-09-13T17:42:03.321984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:41.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262849+0000","last_change":"2024-09-13T17:42:02.347884+0000","last_active":"2024-09-13T17:42:04.262849+0000","last_peered":"2024-09-13T17:42:04.262849+0000","last_clean":"2024-09-13T17:42:04.262849+0000","last_became_active":"2024-09-13T17:42:02.347662+0000","last_became_peered":"2024-09-13T17:42:02.347662+0000","last_unstale":"2024-09-13T17:42:04.262849+0000","last_undegraded":"2024-09-13T17:42:04.262849+0000","last_fullsized":"2024-09-13T17:42:04.262849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:12.394146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321378+0000","last_change":"2024-09-13T17:42:02.362310+0000","last_active":"2024-09-13T17:42:03.321378+0000","last_peered":"2024-09-13T17:42:03.321378+0000","last_clean":"2024-09-13T17:42:03.321378+0000","last_became_active":"2024-09-13T17:42:02.360999+0000","last_became_peered":"2024-09-13T17:42:02.360999+0000","last_unstale":"2024-09-13T17:42:03.321378+0000","last_undegraded":"2024-09-13T17:42:03.321378+0000","last_fullsized":"2024-09-13T17:42:03.321378+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:11.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321607+0000","last_change":"2024-09-13T17:42:02.344968+0000","last_active":"2024-09-13T17:42:03.321607+0000","last_peered":"2024-09-13T17:42:03.321607+0000","last_clean":"2024-09-13T17:42:03.321607+0000","last_became_active":"2024-09-13T17:42:02.344714+0000","last_became_peered":"2024-09-13T17:42:02.344714+0000","last_unstale":"2024-09-13T17:42:03.321607+0000","last_undegraded":"2024-09-13T17:42:03.321607+0000","last_fullsized":"2024-09-13T17:42:03.321607+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912231+0000","last_change":"2024-09-13T17:42:02.345753+0000","last_active":"2024-09-13T17:42:03.912231+0000","last_peered":"2024-09-13T17:42:03.912231+0000","last_clean":"2024-09-13T17:42:03.912231+0000","last_became_active":"2024-09-13T17:42:02.345469+0000","last_became_peered":"2024-09-13T17:42:02.345469+0000","last_unstale":"2024-09-13T17:42:03.912231+0000","last_undegraded":"2024-09-13T17:42:03.912231+0000","last_fullsized":"2024-09-13T17:42:03.912231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912733+0000","last_change":"2024-09-13T17:42:02.363798+0000","last_active":"2024-09-13T17:42:03.912733+0000","last_peered":"2024-09-13T17:42:03.912733+0000","last_clean":"2024-09-13T17:42:03.912733+0000","last_became_active":"2024-09-13T17:42:02.363138+0000","last_became_peered":"2024-09-13T17:42:02.363138+0000","last_unstale":"2024-09-13T17:42:03.912733+0000","last_undegraded":"2024-09-13T17:42:03.912733+0000","last_fullsized":"2024-09-13T17:42:03.912733+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263425+0000","last_change":"2024-09-13T17:42:02.333530+0000","last_active":"2024-09-13T17:42:04.263425+0000","last_peered":"2024-09-13T17:42:04.263425+0000","last_clean":"2024-09-13T17:42:04.263425+0000","last_became_active":"2024-09-13T17:42:02.329725+0000","last_became_peered":"2024-09-13T17:42:02.329725+0000","last_unstale":"2024-09-13T17:42:04.263425+0000","last_undegraded":"2024-09-13T17:42:04.263425+0000","last_fullsized":"2024-09-13T17:42:04.263425+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322104+0000","last_change":"2024-09-13T17:42:02.360444+0000","last_active":"2024-09-13T17:42:03.322104+0000","last_peered":"2024-09-13T17:42:03.322104+0000","last_clean":"2024-09-13T17:42:03.322104+0000","last_became_active":"2024-09-13T17:42:02.357480+0000","last_became_peered":"2024-09-13T17:42:02.357480+0000","last_unstale":"2024-09-13T17:42:03.322104+0000","last_undegraded":"2024-09-13T17:42:03.322104+0000","last_fullsized":"2024-09-13T17:42:03.322104+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:18.088198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322073+0000","last_change":"2024-09-13T17:42:02.360990+0000","last_active":"2024-09-13T17:42:03.322073+0000","last_peered":"2024-09-13T17:42:03.322073+0000","last_clean":"2024-09-13T17:42:03.322073+0000","last_became_active":"2024-09-13T17:42:02.356745+0000","last_became_peered":"2024-09-13T17:42:02.356745+0000","last_unstale":"2024-09-13T17:42:03.322073+0000","last_undegraded":"2024-09-13T17:42:03.322073+0000","last_fullsized":"2024-09-13T17:42:03.322073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321744+0000","last_change":"2024-09-13T17:42:02.351170+0000","last_active":"2024-09-13T17:42:03.321744+0000","last_peered":"2024-09-13T17:42:03.321744+0000","last_clean":"2024-09-13T17:42:03.321744+0000","last_became_active":"2024-09-13T17:42:02.350917+0000","last_became_peered":"2024-09-13T17:42:02.350917+0000","last_unstale":"2024-09-13T17:42:03.321744+0000","last_undegraded":"2024-09-13T17:42:03.321744+0000","last_fullsized":"2024-09-13T17:42:03.321744+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.975775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912828+0000","last_change":"2024-09-13T17:42:02.337517+0000","last_active":"2024-09-13T17:42:03.912828+0000","last_peered":"2024-09-13T17:42:03.912828+0000","last_clean":"2024-09-13T17:42:03.912828+0000","last_became_active":"2024-09-13T17:42:02.336903+0000","last_became_peered":"2024-09-13T17:42:02.336903+0000","last_unstale":"2024-09-13T17:42:03.912828+0000","last_undegraded":"2024-09-13T17:42:03.912828+0000","last_fullsized":"2024-09-13T17:42:03.912828+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:46.917681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912186+0000","last_change":"2024-09-13T17:42:02.364079+0000","last_active":"2024-09-13T17:42:03.912186+0000","last_peered":"2024-09-13T17:42:03.912186+0000","last_clean":"2024-09-13T17:42:03.912186+0000","last_became_active":"2024-09-13T17:42:02.363628+0000","last_became_peered":"2024-09-13T17:42:02.363628+0000","last_unstale":"2024-09-13T17:42:03.912186+0000","last_undegraded":"2024-09-13T17:42:03.912186+0000","last_fullsized":"2024-09-13T17:42:03.912186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263255+0000","last_change":"2024-09-13T17:42:02.333698+0000","last_active":"2024-09-13T17:42:04.263255+0000","last_peered":"2024-09-13T17:42:04.263255+0000","last_clean":"2024-09-13T17:42:04.263255+0000","last_became_active":"2024-09-13T17:42:02.329909+0000","last_became_peered":"2024-09-13T17:42:02.329909+0000","last_unstale":"2024-09-13T17:42:04.263255+0000","last_undegraded":"2024-09-13T17:42:04.263255+0000","last_fullsized":"2024-09-13T17:42:04.263255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264107+0000","last_change":"2024-09-13T17:42:01.989110+0000","last_active":"2024-09-13T17:42:04.264107+0000","last_peered":"2024-09-13T17:42:04.264107+0000","last_clean":"2024-09-13T17:42:04.264107+0000","last_became_active":"2024-09-13T17:42:01.988776+0000","last_became_peered":"2024-09-13T17:42:01.988776+0000","last_unstale":"2024-09-13T17:42:04.264107+0000","last_undegraded":"2024-09-13T17:42:04.264107+0000","last_fullsized":"2024-09-13T17:42:04.264107+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322159+0000","last_change":"2024-09-13T17:42:02.360143+0000","last_active":"2024-09-13T17:42:03.322159+0000","last_peered":"2024-09-13T17:42:03.322159+0000","last_clean":"2024-09-13T17:42:03.322159+0000","last_became_active":"2024-09-13T17:42:02.357112+0000","last_became_peered":"2024-09-13T17:42:02.357112+0000","last_unstale":"2024-09-13T17:42:03.322159+0000","last_undegraded":"2024-09-13T17:42:03.322159+0000","last_fullsized":"2024-09-13T17:42:03.322159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:53.057536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911782+0000","last_change":"2024-09-13T17:42:01.989114+0000","last_active":"2024-09-13T17:42:03.911782+0000","last_peered":"2024-09-13T17:42:03.911782+0000","last_clean":"2024-09-13T17:42:03.911782+0000","last_became_active":"2024-09-13T17:42:01.988536+0000","last_became_peered":"2024-09-13T17:42:01.988536+0000","last_unstale":"2024-09-13T17:42:03.911782+0000","last_undegraded":"2024-09-13T17:42:03.911782+0000","last_fullsized":"2024-09-13T17:42:03.911782+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321624+0000","last_change":"2024-09-13T17:42:01.990295+0000","last_active":"2024-09-13T17:42:03.321624+0000","last_peered":"2024-09-13T17:42:03.321624+0000","last_clean":"2024-09-13T17:42:03.321624+0000","last_became_active":"2024-09-13T17:42:01.989951+0000","last_became_peered":"2024-09-13T17:42:01.989951+0000","last_unstale":"2024-09-13T17:42:03.321624+0000","last_undegraded":"2024-09-13T17:42:03.321624+0000","last_fullsized":"2024-09-13T17:42:03.321624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911808+0000","last_change":"2024-09-13T17:42:02.363919+0000","last_active":"2024-09-13T17:42:03.911808+0000","last_peered":"2024-09-13T17:42:03.911808+0000","last_clean":"2024-09-13T17:42:03.911808+0000","last_became_active":"2024-09-13T17:42:02.363337+0000","last_became_peered":"2024-09-13T17:42:02.363337+0000","last_unstale":"2024-09-13T17:42:03.911808+0000","last_undegraded":"2024-09-13T17:42:03.911808+0000","last_fullsized":"2024-09-13T17:42:03.911808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321842+0000","last_change":"2024-09-13T17:42:01.977030+0000","last_active":"2024-09-13T17:42:03.321842+0000","last_peered":"2024-09-13T17:42:03.321842+0000","last_clean":"2024-09-13T17:42:03.321842+0000","last_became_active":"2024-09-13T17:42:01.976768+0000","last_became_peered":"2024-09-13T17:42:01.976768+0000","last_unstale":"2024-09-13T17:42:03.321842+0000","last_undegraded":"2024-09-13T17:42:03.321842+0000","last_fullsized":"2024-09-13T17:42:03.321842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262490+0000","last_change":"2024-09-13T17:42:02.329537+0000","last_active":"2024-09-13T17:42:04.262490+0000","last_peered":"2024-09-13T17:42:04.262490+0000","last_clean":"2024-09-13T17:42:04.262490+0000","last_became_active":"2024-09-13T17:42:02.329246+0000","last_became_peered":"2024-09-13T17:42:02.329246+0000","last_unstale":"2024-09-13T17:42:04.262490+0000","last_undegraded":"2024-09-13T17:42:04.262490+0000","last_fullsized":"2024-09-13T17:42:04.262490+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264525+0000","last_change":"2024-09-13T17:42:01.974367+0000","last_active":"2024-09-13T17:42:04.264525+0000","last_peered":"2024-09-13T17:42:04.264525+0000","last_clean":"2024-09-13T17:42:04.264525+0000","last_became_active":"2024-09-13T17:42:01.974054+0000","last_became_peered":"2024-09-13T17:42:01.974054+0000","last_unstale":"2024-09-13T17:42:04.264525+0000","last_undegraded":"2024-09-13T17:42:04.264525+0000","last_fullsized":"2024-09-13T17:42:04.264525+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912913+0000","last_change":"2024-09-13T17:42:02.341032+0000","last_active":"2024-09-13T17:42:03.912913+0000","last_peered":"2024-09-13T17:42:03.912913+0000","last_clean":"2024-09-13T17:42:03.912913+0000","last_became_active":"2024-09-13T17:42:02.340777+0000","last_became_peered":"2024-09-13T17:42:02.340777+0000","last_unstale":"2024-09-13T17:42:03.912913+0000","last_undegraded":"2024-09-13T17:42:03.912913+0000","last_fullsized":"2024-09-13T17:42:03.912913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:37.537177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.973354+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.969555+0000","last_became_peered":"2024-09-13T17:42:01.969555+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323123+0000","last_change":"2024-09-13T17:42:02.356982+0000","last_active":"2024-09-13T17:42:03.323123+0000","last_peered":"2024-09-13T17:42:03.323123+0000","last_clean":"2024-09-13T17:42:03.323123+0000","last_became_active":"2024-09-13T17:42:02.354879+0000","last_became_peered":"2024-09-13T17:42:02.354879+0000","last_unstale":"2024-09-13T17:42:03.323123+0000","last_undegraded":"2024-09-13T17:42:03.323123+0000","last_fullsized":"2024-09-13T17:42:03.323123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:00.269193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264141+0000","last_change":"2024-09-13T17:42:01.988113+0000","last_active":"2024-09-13T17:42:04.264141+0000","last_peered":"2024-09-13T17:42:04.264141+0000","last_clean":"2024-09-13T17:42:04.264141+0000","last_became_active":"2024-09-13T17:42:01.987821+0000","last_became_peered":"2024-09-13T17:42:01.987821+0000","last_unstale":"2024-09-13T17:42:04.264141+0000","last_undegraded":"2024-09-13T17:42:04.264141+0000","last_fullsized":"2024-09-13T17:42:04.264141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263852+0000","last_change":"2024-09-13T17:42:02.342353+0000","last_active":"2024-09-13T17:42:04.263852+0000","last_peered":"2024-09-13T17:42:04.263852+0000","last_clean":"2024-09-13T17:42:04.263852+0000","last_became_active":"2024-09-13T17:42:02.336419+0000","last_became_peered":"2024-09-13T17:42:02.336419+0000","last_unstale":"2024-09-13T17:42:04.263852+0000","last_undegraded":"2024-09-13T17:42:04.263852+0000","last_fullsized":"2024-09-13T17:42:04.263852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912570+0000","last_change":"2024-09-13T17:42:01.981005+0000","last_active":"2024-09-13T17:42:03.912570+0000","last_peered":"2024-09-13T17:42:03.912570+0000","last_clean":"2024-09-13T17:42:03.912570+0000","last_became_active":"2024-09-13T17:42:01.978436+0000","last_became_peered":"2024-09-13T17:42:01.978436+0000","last_unstale":"2024-09-13T17:42:03.912570+0000","last_undegraded":"2024-09-13T17:42:03.912570+0000","last_fullsized":"2024-09-13T17:42:03.912570+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322389+0000","last_change":"2024-09-13T17:42:02.354827+0000","last_active":"2024-09-13T17:42:03.322389+0000","last_peered":"2024-09-13T17:42:03.322389+0000","last_clean":"2024-09-13T17:42:03.322389+0000","last_became_active":"2024-09-13T17:42:02.353803+0000","last_became_peered":"2024-09-13T17:42:02.353803+0000","last_unstale":"2024-09-13T17:42:03.322389+0000","last_undegraded":"2024-09-13T17:42:03.322389+0000","last_fullsized":"2024-09-13T17:42:03.322389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:17.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322676+0000","last_change":"2024-09-13T17:42:01.981800+0000","last_active":"2024-09-13T17:42:03.322676+0000","last_peered":"2024-09-13T17:42:03.322676+0000","last_clean":"2024-09-13T17:42:03.322676+0000","last_became_active":"2024-09-13T17:42:01.978579+0000","last_became_peered":"2024-09-13T17:42:01.978579+0000","last_unstale":"2024-09-13T17:42:03.322676+0000","last_undegraded":"2024-09-13T17:42:03.322676+0000","last_fullsized":"2024-09-13T17:42:03.322676+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321641+0000","last_change":"2024-09-13T17:42:02.360084+0000","last_active":"2024-09-13T17:42:03.321641+0000","last_peered":"2024-09-13T17:42:03.321641+0000","last_clean":"2024-09-13T17:42:03.321641+0000","last_became_active":"2024-09-13T17:42:02.358943+0000","last_became_peered":"2024-09-13T17:42:02.358943+0000","last_unstale":"2024-09-13T17:42:03.321641+0000","last_undegraded":"2024-09-13T17:42:03.321641+0000","last_fullsized":"2024-09-13T17:42:03.321641+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:30.309069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911963+0000","last_change":"2024-09-13T17:42:01.980802+0000","last_active":"2024-09-13T17:42:03.911963+0000","last_peered":"2024-09-13T17:42:03.911963+0000","last_clean":"2024-09-13T17:42:03.911963+0000","last_became_active":"2024-09-13T17:42:01.977408+0000","last_became_peered":"2024-09-13T17:42:01.977408+0000","last_unstale":"2024-09-13T17:42:03.911963+0000","last_undegraded":"2024-09-13T17:42:03.911963+0000","last_fullsized":"2024-09-13T17:42:03.911963+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:21.671479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263263+0000","last_change":"2024-09-13T17:42:02.356627+0000","last_active":"2024-09-13T17:42:04.263263+0000","last_peered":"2024-09-13T17:42:04.263263+0000","last_clean":"2024-09-13T17:42:04.263263+0000","last_became_active":"2024-09-13T17:42:02.356381+0000","last_became_peered":"2024-09-13T17:42:02.356381+0000","last_unstale":"2024-09-13T17:42:04.263263+0000","last_undegraded":"2024-09-13T17:42:04.263263+0000","last_fullsized":"2024-09-13T17:42:04.263263+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321850+0000","last_change":"2024-09-13T17:42:01.973218+0000","last_active":"2024-09-13T17:42:03.321850+0000","last_peered":"2024-09-13T17:42:03.321850+0000","last_clean":"2024-09-13T17:42:03.321850+0000","last_became_active":"2024-09-13T17:42:01.972688+0000","last_became_peered":"2024-09-13T17:42:01.972688+0000","last_unstale":"2024-09-13T17:42:03.321850+0000","last_undegraded":"2024-09-13T17:42:03.321850+0000","last_fullsized":"2024-09-13T17:42:03.321850+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262433+0000","last_change":"2024-09-13T17:42:02.348071+0000","last_active":"2024-09-13T17:42:04.262433+0000","last_peered":"2024-09-13T17:42:04.262433+0000","last_clean":"2024-09-13T17:42:04.262433+0000","last_became_active":"2024-09-13T17:42:02.347463+0000","last_became_peered":"2024-09-13T17:42:02.347463+0000","last_unstale":"2024-09-13T17:42:04.262433+0000","last_undegraded":"2024-09-13T17:42:04.262433+0000","last_fullsized":"2024-09-13T17:42:04.262433+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912464+0000","last_change":"2024-09-13T17:42:01.981250+0000","last_active":"2024-09-13T17:42:03.912464+0000","last_peered":"2024-09-13T17:42:03.912464+0000","last_clean":"2024-09-13T17:42:03.912464+0000","last_became_active":"2024-09-13T17:42:01.978674+0000","last_became_peered":"2024-09-13T17:42:01.978674+0000","last_unstale":"2024-09-13T17:42:03.912464+0000","last_undegraded":"2024-09-13T17:42:03.912464+0000","last_fullsized":"2024-09-13T17:42:03.912464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911530+0000","last_change":"2024-09-13T17:42:02.339567+0000","last_active":"2024-09-13T17:42:03.911530+0000","last_peered":"2024-09-13T17:42:03.911530+0000","last_clean":"2024-09-13T17:42:03.911530+0000","last_became_active":"2024-09-13T17:42:02.339304+0000","last_became_peered":"2024-09-13T17:42:02.339304+0000","last_unstale":"2024-09-13T17:42:03.911530+0000","last_undegraded":"2024-09-13T17:42:03.911530+0000","last_fullsized":"2024-09-13T17:42:03.911530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:05:01.291734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323607+0000","last_change":"2024-09-13T17:42:01.973755+0000","last_active":"2024-09-13T17:42:03.323607+0000","last_peered":"2024-09-13T17:42:03.323607+0000","last_clean":"2024-09-13T17:42:03.323607+0000","last_became_active":"2024-09-13T17:42:01.973179+0000","last_became_peered":"2024-09-13T17:42:01.973179+0000","last_unstale":"2024-09-13T17:42:03.323607+0000","last_undegraded":"2024-09-13T17:42:03.323607+0000","last_fullsized":"2024-09-13T17:42:03.323607+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912974+0000","last_change":"2024-09-13T17:42:02.363936+0000","last_active":"2024-09-13T17:42:03.912974+0000","last_peered":"2024-09-13T17:42:03.912974+0000","last_clean":"2024-09-13T17:42:03.912974+0000","last_became_active":"2024-09-13T17:42:02.363316+0000","last_became_peered":"2024-09-13T17:42:02.363316+0000","last_unstale":"2024-09-13T17:42:03.912974+0000","last_undegraded":"2024-09-13T17:42:03.912974+0000","last_fullsized":"2024-09-13T17:42:03.912974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913097+0000","last_change":"2024-09-13T17:42:01.987237+0000","last_active":"2024-09-13T17:42:03.913097+0000","last_peered":"2024-09-13T17:42:03.913097+0000","last_clean":"2024-09-13T17:42:03.913097+0000","last_became_active":"2024-09-13T17:42:01.986684+0000","last_became_peered":"2024-09-13T17:42:01.986684+0000","last_unstale":"2024-09-13T17:42:03.913097+0000","last_undegraded":"2024-09-13T17:42:03.913097+0000","last_fullsized":"2024-09-13T17:42:03.913097+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:27.554217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323169+0000","last_change":"2024-09-13T17:42:02.359350+0000","last_active":"2024-09-13T17:42:03.323169+0000","last_peered":"2024-09-13T17:42:03.323169+0000","last_clean":"2024-09-13T17:42:03.323169+0000","last_became_active":"2024-09-13T17:42:02.357233+0000","last_became_peered":"2024-09-13T17:42:02.357233+0000","last_unstale":"2024-09-13T17:42:03.323169+0000","last_undegraded":"2024-09-13T17:42:03.323169+0000","last_fullsized":"2024-09-13T17:42:03.323169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.541840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264444+0000","last_change":"2024-09-13T17:42:01.970689+0000","last_active":"2024-09-13T17:42:04.264444+0000","last_peered":"2024-09-13T17:42:04.264444+0000","last_clean":"2024-09-13T17:42:04.264444+0000","last_became_active":"2024-09-13T17:42:01.970057+0000","last_became_peered":"2024-09-13T17:42:01.970057+0000","last_unstale":"2024-09-13T17:42:04.264444+0000","last_undegraded":"2024-09-13T17:42:04.264444+0000","last_fullsized":"2024-09-13T17:42:04.264444+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322140+0000","last_change":"2024-09-13T17:42:02.360241+0000","last_active":"2024-09-13T17:42:03.322140+0000","last_peered":"2024-09-13T17:42:03.322140+0000","last_clean":"2024-09-13T17:42:03.322140+0000","last_became_active":"2024-09-13T17:42:02.357150+0000","last_became_peered":"2024-09-13T17:42:02.357150+0000","last_unstale":"2024-09-13T17:42:03.322140+0000","last_undegraded":"2024-09-13T17:42:03.322140+0000","last_fullsized":"2024-09-13T17:42:03.322140+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:36:55.337229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912797+0000","last_change":"2024-09-13T17:42:01.987432+0000","last_active":"2024-09-13T17:42:03.912797+0000","last_peered":"2024-09-13T17:42:03.912797+0000","last_clean":"2024-09-13T17:42:03.912797+0000","last_became_active":"2024-09-13T17:42:01.986822+0000","last_became_peered":"2024-09-13T17:42:01.986822+0000","last_unstale":"2024-09-13T17:42:03.912797+0000","last_undegraded":"2024-09-13T17:42:03.912797+0000","last_fullsized":"2024-09-13T17:42:03.912797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:59.277624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263691+0000","last_change":"2024-09-13T17:42:02.349202+0000","last_active":"2024-09-13T17:42:04.263691+0000","last_peered":"2024-09-13T17:42:04.263691+0000","last_clean":"2024-09-13T17:42:04.263691+0000","last_became_active":"2024-09-13T17:42:02.345781+0000","last_became_peered":"2024-09-13T17:42:02.345781+0000","last_unstale":"2024-09-13T17:42:04.263691+0000","last_undegraded":"2024-09-13T17:42:04.263691+0000","last_fullsized":"2024-09-13T17:42:04.263691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":141,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.317552+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:02.317552+0000","last_peered":"2024-09-13T17:42:02.317552+0000","last_clean":"2024-09-13T17:42:02.317552+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:02.317552+0000","last_undegraded":"2024-09-13T17:42:02.317552+0000","last_fullsized":"2024-09-13T17:42:02.317552+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263958+0000","last_change":"2024-09-13T17:42:01.969286+0000","last_active":"2024-09-13T17:42:04.263958+0000","last_peered":"2024-09-13T17:42:04.263958+0000","last_clean":"2024-09-13T17:42:04.263958+0000","last_became_active":"2024-09-13T17:42:01.968921+0000","last_became_peered":"2024-09-13T17:42:01.968921+0000","last_unstale":"2024-09-13T17:42:04.263958+0000","last_undegraded":"2024-09-13T17:42:04.263958+0000","last_fullsized":"2024-09-13T17:42:04.263958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263120+0000","last_change":"2024-09-13T17:42:02.329801+0000","last_active":"2024-09-13T17:42:04.263120+0000","last_peered":"2024-09-13T17:42:04.263120+0000","last_clean":"2024-09-13T17:42:04.263120+0000","last_became_active":"2024-09-13T17:42:02.329300+0000","last_became_peered":"2024-09-13T17:42:02.329300+0000","last_unstale":"2024-09-13T17:42:04.263120+0000","last_undegraded":"2024-09-13T17:42:04.263120+0000","last_fullsized":"2024-09-13T17:42:04.263120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912706+0000","last_change":"2024-09-13T17:42:01.982284+0000","last_active":"2024-09-13T17:42:03.912706+0000","last_peered":"2024-09-13T17:42:03.912706+0000","last_clean":"2024-09-13T17:42:03.912706+0000","last_became_active":"2024-09-13T17:42:01.981495+0000","last_became_peered":"2024-09-13T17:42:01.981495+0000","last_unstale":"2024-09-13T17:42:03.912706+0000","last_undegraded":"2024-09-13T17:42:03.912706+0000","last_fullsized":"2024-09-13T17:42:03.912706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321462+0000","last_change":"2024-09-13T17:42:02.359429+0000","last_active":"2024-09-13T17:42:03.321462+0000","last_peered":"2024-09-13T17:42:03.321462+0000","last_clean":"2024-09-13T17:42:03.321462+0000","last_became_active":"2024-09-13T17:42:02.357531+0000","last_became_peered":"2024-09-13T17:42:02.357531+0000","last_unstale":"2024-09-13T17:42:03.321462+0000","last_undegraded":"2024-09-13T17:42:03.321462+0000","last_fullsized":"2024-09-13T17:42:03.321462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:43.826159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911994+0000","last_change":"2024-09-13T17:42:01.980799+0000","last_active":"2024-09-13T17:42:03.911994+0000","last_peered":"2024-09-13T17:42:03.911994+0000","last_clean":"2024-09-13T17:42:03.911994+0000","last_became_active":"2024-09-13T17:42:01.977918+0000","last_became_peered":"2024-09-13T17:42:01.977918+0000","last_unstale":"2024-09-13T17:42:03.911994+0000","last_undegraded":"2024-09-13T17:42:03.911994+0000","last_fullsized":"2024-09-13T17:42:03.911994+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263884+0000","last_change":"2024-09-13T17:42:02.339725+0000","last_active":"2024-09-13T17:42:04.263884+0000","last_peered":"2024-09-13T17:42:04.263884+0000","last_clean":"2024-09-13T17:42:04.263884+0000","last_became_active":"2024-09-13T17:42:02.334096+0000","last_became_peered":"2024-09-13T17:42:02.334096+0000","last_unstale":"2024-09-13T17:42:04.263884+0000","last_undegraded":"2024-09-13T17:42:04.263884+0000","last_fullsized":"2024-09-13T17:42:04.263884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322118+0000","last_change":"2024-09-13T17:42:01.982743+0000","last_active":"2024-09-13T17:42:03.322118+0000","last_peered":"2024-09-13T17:42:03.322118+0000","last_clean":"2024-09-13T17:42:03.322118+0000","last_became_active":"2024-09-13T17:42:01.981897+0000","last_became_peered":"2024-09-13T17:42:01.981897+0000","last_unstale":"2024-09-13T17:42:03.322118+0000","last_undegraded":"2024-09-13T17:42:03.322118+0000","last_fullsized":"2024-09-13T17:42:03.322118+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322270+0000","last_change":"2024-09-13T17:42:02.360779+0000","last_active":"2024-09-13T17:42:03.322270+0000","last_peered":"2024-09-13T17:42:03.322270+0000","last_clean":"2024-09-13T17:42:03.322270+0000","last_became_active":"2024-09-13T17:42:02.359476+0000","last_became_peered":"2024-09-13T17:42:02.359476+0000","last_unstale":"2024-09-13T17:42:03.322270+0000","last_undegraded":"2024-09-13T17:42:03.322270+0000","last_fullsized":"2024-09-13T17:42:03.322270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264113+0000","last_change":"2024-09-13T17:42:01.988534+0000","last_active":"2024-09-13T17:42:04.264113+0000","last_peered":"2024-09-13T17:42:04.264113+0000","last_clean":"2024-09-13T17:42:04.264113+0000","last_became_active":"2024-09-13T17:42:01.988254+0000","last_became_peered":"2024-09-13T17:42:01.988254+0000","last_unstale":"2024-09-13T17:42:04.264113+0000","last_undegraded":"2024-09-13T17:42:04.264113+0000","last_fullsized":"2024-09-13T17:42:04.264113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912533+0000","last_change":"2024-09-13T17:42:02.338990+0000","last_active":"2024-09-13T17:42:03.912533+0000","last_peered":"2024-09-13T17:42:03.912533+0000","last_clean":"2024-09-13T17:42:03.912533+0000","last_became_active":"2024-09-13T17:42:02.338737+0000","last_became_peered":"2024-09-13T17:42:02.338737+0000","last_unstale":"2024-09-13T17:42:03.912533+0000","last_undegraded":"2024-09-13T17:42:03.912533+0000","last_fullsized":"2024-09-13T17:42:03.912533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:46:10.901730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264464+0000","last_change":"2024-09-13T17:42:01.966772+0000","last_active":"2024-09-13T17:42:04.264464+0000","last_peered":"2024-09-13T17:42:04.264464+0000","last_clean":"2024-09-13T17:42:04.264464+0000","last_became_active":"2024-09-13T17:42:01.965762+0000","last_became_peered":"2024-09-13T17:42:01.965762+0000","last_unstale":"2024-09-13T17:42:04.264464+0000","last_undegraded":"2024-09-13T17:42:04.264464+0000","last_fullsized":"2024-09-13T17:42:04.264464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264436+0000","last_change":"2024-09-13T17:42:02.340638+0000","last_active":"2024-09-13T17:42:04.264436+0000","last_peered":"2024-09-13T17:42:04.264436+0000","last_clean":"2024-09-13T17:42:04.264436+0000","last_became_active":"2024-09-13T17:42:02.336028+0000","last_became_peered":"2024-09-13T17:42:02.336028+0000","last_unstale":"2024-09-13T17:42:04.264436+0000","last_undegraded":"2024-09-13T17:42:04.264436+0000","last_fullsized":"2024-09-13T17:42:04.264436+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:05.871766+0000","last_change":"2024-09-13T17:42:01.969377+0000","last_active":"2024-09-13T17:42:05.871766+0000","last_peered":"2024-09-13T17:42:05.871766+0000","last_clean":"2024-09-13T17:42:05.871766+0000","last_became_active":"2024-09-13T17:42:01.966888+0000","last_became_peered":"2024-09-13T17:42:01.966888+0000","last_unstale":"2024-09-13T17:42:05.871766+0000","last_undegraded":"2024-09-13T17:42:05.871766+0000","last_fullsized":"2024-09-13T17:42:05.871766+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912530+0000","last_change":"2024-09-13T17:42:02.336164+0000","last_active":"2024-09-13T17:42:03.912530+0000","last_peered":"2024-09-13T17:42:03.912530+0000","last_clean":"2024-09-13T17:42:03.912530+0000","last_became_active":"2024-09-13T17:42:02.335891+0000","last_became_peered":"2024-09-13T17:42:02.335891+0000","last_unstale":"2024-09-13T17:42:03.912530+0000","last_undegraded":"2024-09-13T17:42:03.912530+0000","last_fullsized":"2024-09-13T17:42:03.912530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264680+0000","last_change":"2024-09-13T17:42:01.974797+0000","last_active":"2024-09-13T17:42:04.264680+0000","last_peered":"2024-09-13T17:42:04.264680+0000","last_clean":"2024-09-13T17:42:04.264680+0000","last_became_active":"2024-09-13T17:42:01.971445+0000","last_became_peered":"2024-09-13T17:42:01.971445+0000","last_unstale":"2024-09-13T17:42:04.264680+0000","last_undegraded":"2024-09-13T17:42:04.264680+0000","last_fullsized":"2024-09-13T17:42:04.264680+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322893+0000","last_change":"2024-09-13T17:42:02.354060+0000","last_active":"2024-09-13T17:42:03.322893+0000","last_peered":"2024-09-13T17:42:03.322893+0000","last_clean":"2024-09-13T17:42:03.322893+0000","last_became_active":"2024-09-13T17:42:02.353776+0000","last_became_peered":"2024-09-13T17:42:02.353776+0000","last_unstale":"2024-09-13T17:42:03.322893+0000","last_undegraded":"2024-09-13T17:42:03.322893+0000","last_fullsized":"2024-09-13T17:42:03.322893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.005234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322185+0000","last_change":"2024-09-13T17:42:01.983210+0000","last_active":"2024-09-13T17:42:03.322185+0000","last_peered":"2024-09-13T17:42:03.322185+0000","last_clean":"2024-09-13T17:42:03.322185+0000","last_became_active":"2024-09-13T17:42:01.978757+0000","last_became_peered":"2024-09-13T17:42:01.978757+0000","last_unstale":"2024-09-13T17:42:03.322185+0000","last_undegraded":"2024-09-13T17:42:03.322185+0000","last_fullsized":"2024-09-13T17:42:03.322185+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321316+0000","last_change":"2024-09-13T17:42:02.359330+0000","last_active":"2024-09-13T17:42:03.321316+0000","last_peered":"2024-09-13T17:42:03.321316+0000","last_clean":"2024-09-13T17:42:03.321316+0000","last_became_active":"2024-09-13T17:42:02.358904+0000","last_became_peered":"2024-09-13T17:42:02.358904+0000","last_unstale":"2024-09-13T17:42:03.321316+0000","last_undegraded":"2024-09-13T17:42:03.321316+0000","last_fullsized":"2024-09-13T17:42:03.321316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:45.730743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263264+0000","last_change":"2024-09-13T17:42:01.988623+0000","last_active":"2024-09-13T17:42:04.263264+0000","last_peered":"2024-09-13T17:42:04.263264+0000","last_clean":"2024-09-13T17:42:04.263264+0000","last_became_active":"2024-09-13T17:42:01.988147+0000","last_became_peered":"2024-09-13T17:42:01.988147+0000","last_unstale":"2024-09-13T17:42:04.263264+0000","last_undegraded":"2024-09-13T17:42:04.263264+0000","last_fullsized":"2024-09-13T17:42:04.263264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912143+0000","last_change":"2024-09-13T17:42:02.341209+0000","last_active":"2024-09-13T17:42:03.912143+0000","last_peered":"2024-09-13T17:42:03.912143+0000","last_clean":"2024-09-13T17:42:03.912143+0000","last_became_active":"2024-09-13T17:42:02.340934+0000","last_became_peered":"2024-09-13T17:42:02.340934+0000","last_unstale":"2024-09-13T17:42:03.912143+0000","last_undegraded":"2024-09-13T17:42:03.912143+0000","last_fullsized":"2024-09-13T17:42:03.912143+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:34:26.164972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263495+0000","last_change":"2024-09-13T17:42:01.988051+0000","last_active":"2024-09-13T17:42:04.263495+0000","last_peered":"2024-09-13T17:42:04.263495+0000","last_clean":"2024-09-13T17:42:04.263495+0000","last_became_active":"2024-09-13T17:42:01.987703+0000","last_became_peered":"2024-09-13T17:42:01.987703+0000","last_unstale":"2024-09-13T17:42:04.263495+0000","last_undegraded":"2024-09-13T17:42:04.263495+0000","last_fullsized":"2024-09-13T17:42:04.263495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263566+0000","last_change":"2024-09-13T17:42:02.347875+0000","last_active":"2024-09-13T17:42:04.263566+0000","last_peered":"2024-09-13T17:42:04.263566+0000","last_clean":"2024-09-13T17:42:04.263566+0000","last_became_active":"2024-09-13T17:42:02.347616+0000","last_became_peered":"2024-09-13T17:42:02.347616+0000","last_unstale":"2024-09-13T17:42:04.263566+0000","last_undegraded":"2024-09-13T17:42:04.263566+0000","last_fullsized":"2024-09-13T17:42:04.263566+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.988437+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.987164+0000","last_became_peered":"2024-09-13T17:42:01.987164+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322005+0000","last_change":"2024-09-13T17:42:02.355026+0000","last_active":"2024-09-13T17:42:03.322005+0000","last_peered":"2024-09-13T17:42:03.322005+0000","last_clean":"2024-09-13T17:42:03.322005+0000","last_became_active":"2024-09-13T17:42:02.353935+0000","last_became_peered":"2024-09-13T17:42:02.353935+0000","last_unstale":"2024-09-13T17:42:03.322005+0000","last_undegraded":"2024-09-13T17:42:03.322005+0000","last_fullsized":"2024-09-13T17:42:03.322005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:54.702173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912398+0000","last_change":"2024-09-13T17:42:02.363456+0000","last_active":"2024-09-13T17:42:03.912398+0000","last_peered":"2024-09-13T17:42:03.912398+0000","last_clean":"2024-09-13T17:42:03.912398+0000","last_became_active":"2024-09-13T17:42:02.363171+0000","last_became_peered":"2024-09-13T17:42:02.363171+0000","last_unstale":"2024-09-13T17:42:03.912398+0000","last_undegraded":"2024-09-13T17:42:03.912398+0000","last_fullsized":"2024-09-13T17:42:03.912398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322259+0000","last_change":"2024-09-13T17:42:01.977493+0000","last_active":"2024-09-13T17:42:03.322259+0000","last_peered":"2024-09-13T17:42:03.322259+0000","last_clean":"2024-09-13T17:42:03.322259+0000","last_became_active":"2024-09-13T17:42:01.976932+0000","last_became_peered":"2024-09-13T17:42:01.976932+0000","last_unstale":"2024-09-13T17:42:03.322259+0000","last_undegraded":"2024-09-13T17:42:03.322259+0000","last_fullsized":"2024-09-13T17:42:03.322259+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263697+0000","last_change":"2024-09-13T17:42:02.348464+0000","last_active":"2024-09-13T17:42:04.263697+0000","last_peered":"2024-09-13T17:42:04.263697+0000","last_clean":"2024-09-13T17:42:04.263697+0000","last_became_active":"2024-09-13T17:42:02.348208+0000","last_became_peered":"2024-09-13T17:42:02.348208+0000","last_unstale":"2024-09-13T17:42:04.263697+0000","last_undegraded":"2024-09-13T17:42:04.263697+0000","last_fullsized":"2024-09-13T17:42:04.263697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322576+0000","last_change":"2024-09-13T17:42:01.986020+0000","last_active":"2024-09-13T17:42:03.322576+0000","last_peered":"2024-09-13T17:42:03.322576+0000","last_clean":"2024-09-13T17:42:03.322576+0000","last_became_active":"2024-09-13T17:42:01.985663+0000","last_became_peered":"2024-09-13T17:42:01.985663+0000","last_unstale":"2024-09-13T17:42:03.322576+0000","last_undegraded":"2024-09-13T17:42:03.322576+0000","last_fullsized":"2024-09-13T17:42:03.322576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:12.942087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323065+0000","last_change":"2024-09-13T17:42:02.359950+0000","last_active":"2024-09-13T17:42:03.323065+0000","last_peered":"2024-09-13T17:42:03.323065+0000","last_clean":"2024-09-13T17:42:03.323065+0000","last_became_active":"2024-09-13T17:42:02.358808+0000","last_became_peered":"2024-09-13T17:42:02.358808+0000","last_unstale":"2024-09-13T17:42:03.323065+0000","last_undegraded":"2024-09-13T17:42:03.323065+0000","last_fullsized":"2024-09-13T17:42:03.323065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:37.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913042+0000","last_change":"2024-09-13T17:42:01.987023+0000","last_active":"2024-09-13T17:42:03.913042+0000","last_peered":"2024-09-13T17:42:03.913042+0000","last_clean":"2024-09-13T17:42:03.913042+0000","last_became_active":"2024-09-13T17:42:01.986220+0000","last_became_peered":"2024-09-13T17:42:01.986220+0000","last_unstale":"2024-09-13T17:42:03.913042+0000","last_undegraded":"2024-09-13T17:42:03.913042+0000","last_fullsized":"2024-09-13T17:42:03.913042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:12.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912685+0000","last_change":"2024-09-13T17:42:02.338763+0000","last_active":"2024-09-13T17:42:03.912685+0000","last_peered":"2024-09-13T17:42:03.912685+0000","last_clean":"2024-09-13T17:42:03.912685+0000","last_became_active":"2024-09-13T17:42:02.338494+0000","last_became_peered":"2024-09-13T17:42:02.338494+0000","last_unstale":"2024-09-13T17:42:03.912685+0000","last_undegraded":"2024-09-13T17:42:03.912685+0000","last_fullsized":"2024-09-13T17:42:03.912685+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323462+0000","last_change":"2024-09-13T17:42:01.986108+0000","last_active":"2024-09-13T17:42:03.323462+0000","last_peered":"2024-09-13T17:42:03.323462+0000","last_clean":"2024-09-13T17:42:03.323462+0000","last_became_active":"2024-09-13T17:42:01.985698+0000","last_became_peered":"2024-09-13T17:42:01.985698+0000","last_unstale":"2024-09-13T17:42:03.323462+0000","last_undegraded":"2024-09-13T17:42:03.323462+0000","last_fullsized":"2024-09-13T17:42:03.323462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":42001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":633029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":741376,"data_stored":631429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:13.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:13.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.377+0000 7f89a78c3640 1 Processor -- start 2024-09-13T17:42:13.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.377+0000 7f89a78c3640 1 -- start start 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a78c3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f89a0137490 0x7f89a0139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a78c3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a0071670 con 0x7f89a0134670 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a78c3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a0071360 con 0x7f89a0137490 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f89a0137490 0x7f89a0139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f89a0137490 0x7f89a0139880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39410/0 (socket says 172.21.15.96:39410) 2024-09-13T17:42:13.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 -- 172.21.15.96:0/779171999 learned_addr learned my addr 172.21.15.96:0/779171999 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:13.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a565f640 1 --1- 172.21.15.96:0/779171999 >> v1:172.21.15.96:6789/0 conn(0x7f89a0134670 0x7f89a0134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58940/0 (socket says 172.21.15.96:58940) 2024-09-13T17:42:13.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 -- 172.21.15.96:0/779171999 >> v1:172.21.15.96:6789/0 conn(0x7f89a0134670 legacy=0x7f89a0134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 -- 172.21.15.96:0/779171999 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00a7200 con 0x7f89a0137490 2024-09-13T17:42:13.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f8997fff640 1 -- 172.21.15.96:0/779171999 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003377742 0 0) 0x7f89a0071670 con 0x7f89a0134670 2024-09-13T17:42:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f89a4e5e640 1 --2- 172.21.15.96:0/779171999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0137490 0x7f89a0139880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f89900097f0 tx=0x7f899001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ed5e0dafc1e3b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f8997fff640 1 -- 172.21.15.96:0/779171999 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89900ab530 con 0x7f89a0137490 2024-09-13T17:42:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f8997fff640 1 -- 172.21.15.96:0/779171999 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89900a7050 con 0x7f89a0137490 2024-09-13T17:42:13.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.381+0000 7f8997fff640 1 -- 172.21.15.96:0/779171999 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89900ab7e0 con 0x7f89a0137490 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/779171999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0137490 msgr2=0x7f89a0139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 --2- 172.21.15.96:0/779171999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0137490 0x7f89a0139880 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f89900097f0 tx=0x7f899001c450 comp rx=0 tx=0).stop 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/779171999 shutdown_connections 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 --2- 172.21.15.96:0/779171999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0137490 0x7f89a0139880 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/779171999 >> 172.21.15.96:0/779171999 conn(0x7f89a0098a60 msgr2=0x7f89a009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/779171999 shutdown_connections 2024-09-13T17:42:13.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/779171999 wait complete. 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 Processor -- start 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- start start 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a00a0650 con 0x7f89a0134670 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a565f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a565f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39424/0 (socket says 172.21.15.96:39424) 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a565f640 1 -- 172.21.15.96:0/2181120283 learned_addr learned my addr 172.21.15.96:0/2181120283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a565f640 1 -- 172.21.15.96:0/2181120283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89900094d0 con 0x7f89a0134670 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a565f640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c420 tx=0x7f899c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f899c0074d0 con 0x7f89a0134670 2024-09-13T17:42:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c007630 con 0x7f89a0134670 2024-09-13T17:42:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a009fbf0 con 0x7f89a0134670 2024-09-13T17:42:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f899c004960 con 0x7f89a0134670 2024-09-13T17:42:13.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a009f8a0 con 0x7f89a0134670 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f899c004ac0 con 0x7f89a0134670 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a0134a50 con 0x7f89a0134670 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 0x7f8984037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f899c046c40 con 0x7f89a0134670 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a4e5e640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 0x7f8984037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.385+0000 7f89a4e5e640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 0x7f8984037910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89900ad000 tx=0x7f8990008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.393+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f899c01acd0 con 0x7f89a0134670 2024-09-13T17:42:13.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.533+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f89a009d520 con 0x7f8984035450 2024-09-13T17:42:13.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.541+0000 7f8995ffb640 1 -- 172.21.15.96:0/2181120283 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247338 (secure 0 0 0) 0x7f89900ab660 con 0x7f8984035450 2024-09-13T17:42:13.543 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:13.544 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:13.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 msgr2=0x7f8984037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 0x7f8984037910 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f89900ad000 tx=0x7f8990008040 comp rx=0 tx=0).stop 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 msgr2=0x7f89a01cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c420 tx=0x7f899c00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 shutdown_connections 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8984035450 0x7f8984037910 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 --2- 172.21.15.96:0/2181120283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f89a0134670 0x7f89a01cda40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 >> 172.21.15.96:0/2181120283 conn(0x7f89a0098a60 msgr2=0x7f89a012dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 shutdown_connections 2024-09-13T17:42:13.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.545+0000 7f89a78c3640 1 -- 172.21.15.96:0/2181120283 wait complete. 2024-09-13T17:42:13.559 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-13T17:42:11.798043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":182,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82668,"kb_used_data":1608,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032852,"statfs":{"total":289910292480,"available":289825640448,"internally_reserved":0,"allocated":1646592,"data_stored":1306459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.482736"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322958+0000","last_change":"2024-09-13T17:42:02.360541+0000","last_active":"2024-09-13T17:42:03.322958+0000","last_peered":"2024-09-13T17:42:03.322958+0000","last_clean":"2024-09-13T17:42:03.322958+0000","last_became_active":"2024-09-13T17:42:02.359334+0000","last_became_peered":"2024-09-13T17:42:02.359334+0000","last_unstale":"2024-09-13T17:42:03.322958+0000","last_undegraded":"2024-09-13T17:42:03.322958+0000","last_fullsized":"2024-09-13T17:42:03.322958+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:42.629895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264089+0000","last_change":"2024-09-13T17:42:02.357141+0000","last_active":"2024-09-13T17:42:04.264089+0000","last_peered":"2024-09-13T17:42:04.264089+0000","last_clean":"2024-09-13T17:42:04.264089+0000","last_became_active":"2024-09-13T17:42:02.356712+0000","last_became_peered":"2024-09-13T17:42:02.356712+0000","last_unstale":"2024-09-13T17:42:04.264089+0000","last_undegraded":"2024-09-13T17:42:04.264089+0000","last_fullsized":"2024-09-13T17:42:04.264089+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322398+0000","last_change":"2024-09-13T17:42:02.353311+0000","last_active":"2024-09-13T17:42:03.322398+0000","last_peered":"2024-09-13T17:42:03.322398+0000","last_clean":"2024-09-13T17:42:03.322398+0000","last_became_active":"2024-09-13T17:42:02.351787+0000","last_became_peered":"2024-09-13T17:42:02.351787+0000","last_unstale":"2024-09-13T17:42:03.322398+0000","last_undegraded":"2024-09-13T17:42:03.322398+0000","last_fullsized":"2024-09-13T17:42:03.322398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:14:35.554439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912292+0000","last_change":"2024-09-13T17:42:02.364812+0000","last_active":"2024-09-13T17:42:03.912292+0000","last_peered":"2024-09-13T17:42:03.912292+0000","last_clean":"2024-09-13T17:42:03.912292+0000","last_became_active":"2024-09-13T17:42:02.364256+0000","last_became_peered":"2024-09-13T17:42:02.364256+0000","last_unstale":"2024-09-13T17:42:03.912292+0000","last_undegraded":"2024-09-13T17:42:03.912292+0000","last_fullsized":"2024-09-13T17:42:03.912292+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322703+0000","last_change":"2024-09-13T17:42:02.359338+0000","last_active":"2024-09-13T17:42:03.322703+0000","last_peered":"2024-09-13T17:42:03.322703+0000","last_clean":"2024-09-13T17:42:03.322703+0000","last_became_active":"2024-09-13T17:42:02.355220+0000","last_became_peered":"2024-09-13T17:42:02.355220+0000","last_unstale":"2024-09-13T17:42:03.322703+0000","last_undegraded":"2024-09-13T17:42:03.322703+0000","last_fullsized":"2024-09-13T17:42:03.322703+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:42:41.277636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911521+0000","last_change":"2024-09-13T17:42:02.364229+0000","last_active":"2024-09-13T17:42:03.911521+0000","last_peered":"2024-09-13T17:42:03.911521+0000","last_clean":"2024-09-13T17:42:03.911521+0000","last_became_active":"2024-09-13T17:42:02.363257+0000","last_became_peered":"2024-09-13T17:42:02.363257+0000","last_unstale":"2024-09-13T17:42:03.911521+0000","last_undegraded":"2024-09-13T17:42:03.911521+0000","last_fullsized":"2024-09-13T17:42:03.911521+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321735+0000","last_change":"2024-09-13T17:42:02.353689+0000","last_active":"2024-09-13T17:42:03.321735+0000","last_peered":"2024-09-13T17:42:03.321735+0000","last_clean":"2024-09-13T17:42:03.321735+0000","last_became_active":"2024-09-13T17:42:02.353466+0000","last_became_peered":"2024-09-13T17:42:02.353466+0000","last_unstale":"2024-09-13T17:42:03.321735+0000","last_undegraded":"2024-09-13T17:42:03.321735+0000","last_fullsized":"2024-09-13T17:42:03.321735+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:15:23.813864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911081+0000","last_change":"2024-09-13T17:42:02.351016+0000","last_active":"2024-09-13T17:42:03.911081+0000","last_peered":"2024-09-13T17:42:03.911081+0000","last_clean":"2024-09-13T17:42:03.911081+0000","last_became_active":"2024-09-13T17:42:02.349745+0000","last_became_peered":"2024-09-13T17:42:02.349745+0000","last_unstale":"2024-09-13T17:42:03.911081+0000","last_undegraded":"2024-09-13T17:42:03.911081+0000","last_fullsized":"2024-09-13T17:42:03.911081+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264317+0000","last_change":"2024-09-13T17:42:02.348417+0000","last_active":"2024-09-13T17:42:04.264317+0000","last_peered":"2024-09-13T17:42:04.264317+0000","last_clean":"2024-09-13T17:42:04.264317+0000","last_became_active":"2024-09-13T17:42:02.348073+0000","last_became_peered":"2024-09-13T17:42:02.348073+0000","last_unstale":"2024-09-13T17:42:04.264317+0000","last_undegraded":"2024-09-13T17:42:04.264317+0000","last_fullsized":"2024-09-13T17:42:04.264317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322820+0000","last_change":"2024-09-13T17:42:02.348661+0000","last_active":"2024-09-13T17:42:03.322820+0000","last_peered":"2024-09-13T17:42:03.322820+0000","last_clean":"2024-09-13T17:42:03.322820+0000","last_became_active":"2024-09-13T17:42:02.348395+0000","last_became_peered":"2024-09-13T17:42:02.348395+0000","last_unstale":"2024-09-13T17:42:03.322820+0000","last_undegraded":"2024-09-13T17:42:03.322820+0000","last_fullsized":"2024-09-13T17:42:03.322820+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:27:44.538462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322196+0000","last_change":"2024-09-13T17:42:02.355321+0000","last_active":"2024-09-13T17:42:03.322196+0000","last_peered":"2024-09-13T17:42:03.322196+0000","last_clean":"2024-09-13T17:42:03.322196+0000","last_became_active":"2024-09-13T17:42:02.355024+0000","last_became_peered":"2024-09-13T17:42:02.355024+0000","last_unstale":"2024-09-13T17:42:03.322196+0000","last_undegraded":"2024-09-13T17:42:03.322196+0000","last_fullsized":"2024-09-13T17:42:03.322196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:52:24.061971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321999+0000","last_change":"2024-09-13T17:42:02.360863+0000","last_active":"2024-09-13T17:42:03.321999+0000","last_peered":"2024-09-13T17:42:03.321999+0000","last_clean":"2024-09-13T17:42:03.321999+0000","last_became_active":"2024-09-13T17:42:02.359969+0000","last_became_peered":"2024-09-13T17:42:02.359969+0000","last_unstale":"2024-09-13T17:42:03.321999+0000","last_undegraded":"2024-09-13T17:42:03.321999+0000","last_fullsized":"2024-09-13T17:42:03.321999+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:37:18.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322818+0000","last_change":"2024-09-13T17:42:02.357373+0000","last_active":"2024-09-13T17:42:03.322818+0000","last_peered":"2024-09-13T17:42:03.322818+0000","last_clean":"2024-09-13T17:42:03.322818+0000","last_became_active":"2024-09-13T17:42:02.354117+0000","last_became_peered":"2024-09-13T17:42:02.354117+0000","last_unstale":"2024-09-13T17:42:03.322818+0000","last_undegraded":"2024-09-13T17:42:03.322818+0000","last_fullsized":"2024-09-13T17:42:03.322818+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:34:28.692862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321544+0000","last_change":"2024-09-13T17:42:02.359487+0000","last_active":"2024-09-13T17:42:03.321544+0000","last_peered":"2024-09-13T17:42:03.321544+0000","last_clean":"2024-09-13T17:42:03.321544+0000","last_became_active":"2024-09-13T17:42:02.358942+0000","last_became_peered":"2024-09-13T17:42:02.358942+0000","last_unstale":"2024-09-13T17:42:03.321544+0000","last_undegraded":"2024-09-13T17:42:03.321544+0000","last_fullsized":"2024-09-13T17:42:03.321544+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:25.373355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263024+0000","last_change":"2024-09-13T17:42:02.349044+0000","last_active":"2024-09-13T17:42:04.263024+0000","last_peered":"2024-09-13T17:42:04.263024+0000","last_clean":"2024-09-13T17:42:04.263024+0000","last_became_active":"2024-09-13T17:42:02.345647+0000","last_became_peered":"2024-09-13T17:42:02.345647+0000","last_unstale":"2024-09-13T17:42:04.263024+0000","last_undegraded":"2024-09-13T17:42:04.263024+0000","last_fullsized":"2024-09-13T17:42:04.263024+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911223+0000","last_change":"2024-09-13T17:42:02.363617+0000","last_active":"2024-09-13T17:42:03.911223+0000","last_peered":"2024-09-13T17:42:03.911223+0000","last_clean":"2024-09-13T17:42:03.911223+0000","last_became_active":"2024-09-13T17:42:02.362957+0000","last_became_peered":"2024-09-13T17:42:02.362957+0000","last_unstale":"2024-09-13T17:42:03.911223+0000","last_undegraded":"2024-09-13T17:42:03.911223+0000","last_fullsized":"2024-09-13T17:42:03.911223+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264682+0000","last_change":"2024-09-13T17:42:02.349237+0000","last_active":"2024-09-13T17:42:04.264682+0000","last_peered":"2024-09-13T17:42:04.264682+0000","last_clean":"2024-09-13T17:42:04.264682+0000","last_became_active":"2024-09-13T17:42:02.348179+0000","last_became_peered":"2024-09-13T17:42:02.348179+0000","last_unstale":"2024-09-13T17:42:04.264682+0000","last_undegraded":"2024-09-13T17:42:04.264682+0000","last_fullsized":"2024-09-13T17:42:04.264682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912431+0000","last_change":"2024-09-13T17:42:02.363954+0000","last_active":"2024-09-13T17:42:03.912431+0000","last_peered":"2024-09-13T17:42:03.912431+0000","last_clean":"2024-09-13T17:42:03.912431+0000","last_became_active":"2024-09-13T17:42:02.363490+0000","last_became_peered":"2024-09-13T17:42:02.363490+0000","last_unstale":"2024-09-13T17:42:03.912431+0000","last_undegraded":"2024-09-13T17:42:03.912431+0000","last_fullsized":"2024-09-13T17:42:03.912431+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:41:00.370809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322498+0000","last_change":"2024-09-13T17:42:02.344432+0000","last_active":"2024-09-13T17:42:03.322498+0000","last_peered":"2024-09-13T17:42:03.322498+0000","last_clean":"2024-09-13T17:42:03.322498+0000","last_became_active":"2024-09-13T17:42:02.344155+0000","last_became_peered":"2024-09-13T17:42:02.344155+0000","last_unstale":"2024-09-13T17:42:03.322498+0000","last_undegraded":"2024-09-13T17:42:03.322498+0000","last_fullsized":"2024-09-13T17:42:03.322498+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:02:24.986220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321991+0000","last_change":"2024-09-13T17:42:02.360682+0000","last_active":"2024-09-13T17:42:03.321991+0000","last_peered":"2024-09-13T17:42:03.321991+0000","last_clean":"2024-09-13T17:42:03.321991+0000","last_became_active":"2024-09-13T17:42:02.356733+0000","last_became_peered":"2024-09-13T17:42:02.356733+0000","last_unstale":"2024-09-13T17:42:03.321991+0000","last_undegraded":"2024-09-13T17:42:03.321991+0000","last_fullsized":"2024-09-13T17:42:03.321991+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:43.147144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321984+0000","last_change":"2024-09-13T17:42:02.360599+0000","last_active":"2024-09-13T17:42:03.321984+0000","last_peered":"2024-09-13T17:42:03.321984+0000","last_clean":"2024-09-13T17:42:03.321984+0000","last_became_active":"2024-09-13T17:42:02.357562+0000","last_became_peered":"2024-09-13T17:42:02.357562+0000","last_unstale":"2024-09-13T17:42:03.321984+0000","last_undegraded":"2024-09-13T17:42:03.321984+0000","last_fullsized":"2024-09-13T17:42:03.321984+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:37:41.929332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262849+0000","last_change":"2024-09-13T17:42:02.347884+0000","last_active":"2024-09-13T17:42:04.262849+0000","last_peered":"2024-09-13T17:42:04.262849+0000","last_clean":"2024-09-13T17:42:04.262849+0000","last_became_active":"2024-09-13T17:42:02.347662+0000","last_became_peered":"2024-09-13T17:42:02.347662+0000","last_unstale":"2024-09-13T17:42:04.262849+0000","last_undegraded":"2024-09-13T17:42:04.262849+0000","last_fullsized":"2024-09-13T17:42:04.262849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:12.394146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321378+0000","last_change":"2024-09-13T17:42:02.362310+0000","last_active":"2024-09-13T17:42:03.321378+0000","last_peered":"2024-09-13T17:42:03.321378+0000","last_clean":"2024-09-13T17:42:03.321378+0000","last_became_active":"2024-09-13T17:42:02.360999+0000","last_became_peered":"2024-09-13T17:42:02.360999+0000","last_unstale":"2024-09-13T17:42:03.321378+0000","last_undegraded":"2024-09-13T17:42:03.321378+0000","last_fullsized":"2024-09-13T17:42:03.321378+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:22:11.319747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321607+0000","last_change":"2024-09-13T17:42:02.344968+0000","last_active":"2024-09-13T17:42:03.321607+0000","last_peered":"2024-09-13T17:42:03.321607+0000","last_clean":"2024-09-13T17:42:03.321607+0000","last_became_active":"2024-09-13T17:42:02.344714+0000","last_became_peered":"2024-09-13T17:42:02.344714+0000","last_unstale":"2024-09-13T17:42:03.321607+0000","last_undegraded":"2024-09-13T17:42:03.321607+0000","last_fullsized":"2024-09-13T17:42:03.321607+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:51.319287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912231+0000","last_change":"2024-09-13T17:42:02.345753+0000","last_active":"2024-09-13T17:42:03.912231+0000","last_peered":"2024-09-13T17:42:03.912231+0000","last_clean":"2024-09-13T17:42:03.912231+0000","last_became_active":"2024-09-13T17:42:02.345469+0000","last_became_peered":"2024-09-13T17:42:02.345469+0000","last_unstale":"2024-09-13T17:42:03.912231+0000","last_undegraded":"2024-09-13T17:42:03.912231+0000","last_fullsized":"2024-09-13T17:42:03.912231+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912733+0000","last_change":"2024-09-13T17:42:02.363798+0000","last_active":"2024-09-13T17:42:03.912733+0000","last_peered":"2024-09-13T17:42:03.912733+0000","last_clean":"2024-09-13T17:42:03.912733+0000","last_became_active":"2024-09-13T17:42:02.363138+0000","last_became_peered":"2024-09-13T17:42:02.363138+0000","last_unstale":"2024-09-13T17:42:03.912733+0000","last_undegraded":"2024-09-13T17:42:03.912733+0000","last_fullsized":"2024-09-13T17:42:03.912733+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263425+0000","last_change":"2024-09-13T17:42:02.333530+0000","last_active":"2024-09-13T17:42:04.263425+0000","last_peered":"2024-09-13T17:42:04.263425+0000","last_clean":"2024-09-13T17:42:04.263425+0000","last_became_active":"2024-09-13T17:42:02.329725+0000","last_became_peered":"2024-09-13T17:42:02.329725+0000","last_unstale":"2024-09-13T17:42:04.263425+0000","last_undegraded":"2024-09-13T17:42:04.263425+0000","last_fullsized":"2024-09-13T17:42:04.263425+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322104+0000","last_change":"2024-09-13T17:42:02.360444+0000","last_active":"2024-09-13T17:42:03.322104+0000","last_peered":"2024-09-13T17:42:03.322104+0000","last_clean":"2024-09-13T17:42:03.322104+0000","last_became_active":"2024-09-13T17:42:02.357480+0000","last_became_peered":"2024-09-13T17:42:02.357480+0000","last_unstale":"2024-09-13T17:42:03.322104+0000","last_undegraded":"2024-09-13T17:42:03.322104+0000","last_fullsized":"2024-09-13T17:42:03.322104+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:18.088198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322073+0000","last_change":"2024-09-13T17:42:02.360990+0000","last_active":"2024-09-13T17:42:03.322073+0000","last_peered":"2024-09-13T17:42:03.322073+0000","last_clean":"2024-09-13T17:42:03.322073+0000","last_became_active":"2024-09-13T17:42:02.356745+0000","last_became_peered":"2024-09-13T17:42:02.356745+0000","last_unstale":"2024-09-13T17:42:03.322073+0000","last_undegraded":"2024-09-13T17:42:03.322073+0000","last_fullsized":"2024-09-13T17:42:03.322073+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:34:49.854378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321744+0000","last_change":"2024-09-13T17:42:02.351170+0000","last_active":"2024-09-13T17:42:03.321744+0000","last_peered":"2024-09-13T17:42:03.321744+0000","last_clean":"2024-09-13T17:42:03.321744+0000","last_became_active":"2024-09-13T17:42:02.350917+0000","last_became_peered":"2024-09-13T17:42:02.350917+0000","last_unstale":"2024-09-13T17:42:03.321744+0000","last_undegraded":"2024-09-13T17:42:03.321744+0000","last_fullsized":"2024-09-13T17:42:03.321744+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:55.975775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912828+0000","last_change":"2024-09-13T17:42:02.337517+0000","last_active":"2024-09-13T17:42:03.912828+0000","last_peered":"2024-09-13T17:42:03.912828+0000","last_clean":"2024-09-13T17:42:03.912828+0000","last_became_active":"2024-09-13T17:42:02.336903+0000","last_became_peered":"2024-09-13T17:42:02.336903+0000","last_unstale":"2024-09-13T17:42:03.912828+0000","last_undegraded":"2024-09-13T17:42:03.912828+0000","last_fullsized":"2024-09-13T17:42:03.912828+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:46.917681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912186+0000","last_change":"2024-09-13T17:42:02.364079+0000","last_active":"2024-09-13T17:42:03.912186+0000","last_peered":"2024-09-13T17:42:03.912186+0000","last_clean":"2024-09-13T17:42:03.912186+0000","last_became_active":"2024-09-13T17:42:02.363628+0000","last_became_peered":"2024-09-13T17:42:02.363628+0000","last_unstale":"2024-09-13T17:42:03.912186+0000","last_undegraded":"2024-09-13T17:42:03.912186+0000","last_fullsized":"2024-09-13T17:42:03.912186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.052329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263255+0000","last_change":"2024-09-13T17:42:02.333698+0000","last_active":"2024-09-13T17:42:04.263255+0000","last_peered":"2024-09-13T17:42:04.263255+0000","last_clean":"2024-09-13T17:42:04.263255+0000","last_became_active":"2024-09-13T17:42:02.329909+0000","last_became_peered":"2024-09-13T17:42:02.329909+0000","last_unstale":"2024-09-13T17:42:04.263255+0000","last_undegraded":"2024-09-13T17:42:04.263255+0000","last_fullsized":"2024-09-13T17:42:04.263255+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264107+0000","last_change":"2024-09-13T17:42:01.989110+0000","last_active":"2024-09-13T17:42:04.264107+0000","last_peered":"2024-09-13T17:42:04.264107+0000","last_clean":"2024-09-13T17:42:04.264107+0000","last_became_active":"2024-09-13T17:42:01.988776+0000","last_became_peered":"2024-09-13T17:42:01.988776+0000","last_unstale":"2024-09-13T17:42:04.264107+0000","last_undegraded":"2024-09-13T17:42:04.264107+0000","last_fullsized":"2024-09-13T17:42:04.264107+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322159+0000","last_change":"2024-09-13T17:42:02.360143+0000","last_active":"2024-09-13T17:42:03.322159+0000","last_peered":"2024-09-13T17:42:03.322159+0000","last_clean":"2024-09-13T17:42:03.322159+0000","last_became_active":"2024-09-13T17:42:02.357112+0000","last_became_peered":"2024-09-13T17:42:02.357112+0000","last_unstale":"2024-09-13T17:42:03.322159+0000","last_undegraded":"2024-09-13T17:42:03.322159+0000","last_fullsized":"2024-09-13T17:42:03.322159+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:40:53.057536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911782+0000","last_change":"2024-09-13T17:42:01.989114+0000","last_active":"2024-09-13T17:42:03.911782+0000","last_peered":"2024-09-13T17:42:03.911782+0000","last_clean":"2024-09-13T17:42:03.911782+0000","last_became_active":"2024-09-13T17:42:01.988536+0000","last_became_peered":"2024-09-13T17:42:01.988536+0000","last_unstale":"2024-09-13T17:42:03.911782+0000","last_undegraded":"2024-09-13T17:42:03.911782+0000","last_fullsized":"2024-09-13T17:42:03.911782+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321624+0000","last_change":"2024-09-13T17:42:01.990295+0000","last_active":"2024-09-13T17:42:03.321624+0000","last_peered":"2024-09-13T17:42:03.321624+0000","last_clean":"2024-09-13T17:42:03.321624+0000","last_became_active":"2024-09-13T17:42:01.989951+0000","last_became_peered":"2024-09-13T17:42:01.989951+0000","last_unstale":"2024-09-13T17:42:03.321624+0000","last_undegraded":"2024-09-13T17:42:03.321624+0000","last_fullsized":"2024-09-13T17:42:03.321624+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911808+0000","last_change":"2024-09-13T17:42:02.363919+0000","last_active":"2024-09-13T17:42:03.911808+0000","last_peered":"2024-09-13T17:42:03.911808+0000","last_clean":"2024-09-13T17:42:03.911808+0000","last_became_active":"2024-09-13T17:42:02.363337+0000","last_became_peered":"2024-09-13T17:42:02.363337+0000","last_unstale":"2024-09-13T17:42:03.911808+0000","last_undegraded":"2024-09-13T17:42:03.911808+0000","last_fullsized":"2024-09-13T17:42:03.911808+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321842+0000","last_change":"2024-09-13T17:42:01.977030+0000","last_active":"2024-09-13T17:42:03.321842+0000","last_peered":"2024-09-13T17:42:03.321842+0000","last_clean":"2024-09-13T17:42:03.321842+0000","last_became_active":"2024-09-13T17:42:01.976768+0000","last_became_peered":"2024-09-13T17:42:01.976768+0000","last_unstale":"2024-09-13T17:42:03.321842+0000","last_undegraded":"2024-09-13T17:42:03.321842+0000","last_fullsized":"2024-09-13T17:42:03.321842+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262490+0000","last_change":"2024-09-13T17:42:02.329537+0000","last_active":"2024-09-13T17:42:04.262490+0000","last_peered":"2024-09-13T17:42:04.262490+0000","last_clean":"2024-09-13T17:42:04.262490+0000","last_became_active":"2024-09-13T17:42:02.329246+0000","last_became_peered":"2024-09-13T17:42:02.329246+0000","last_unstale":"2024-09-13T17:42:04.262490+0000","last_undegraded":"2024-09-13T17:42:04.262490+0000","last_fullsized":"2024-09-13T17:42:04.262490+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:42:52.019888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264525+0000","last_change":"2024-09-13T17:42:01.974367+0000","last_active":"2024-09-13T17:42:04.264525+0000","last_peered":"2024-09-13T17:42:04.264525+0000","last_clean":"2024-09-13T17:42:04.264525+0000","last_became_active":"2024-09-13T17:42:01.974054+0000","last_became_peered":"2024-09-13T17:42:01.974054+0000","last_unstale":"2024-09-13T17:42:04.264525+0000","last_undegraded":"2024-09-13T17:42:04.264525+0000","last_fullsized":"2024-09-13T17:42:04.264525+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912913+0000","last_change":"2024-09-13T17:42:02.341032+0000","last_active":"2024-09-13T17:42:03.912913+0000","last_peered":"2024-09-13T17:42:03.912913+0000","last_clean":"2024-09-13T17:42:03.912913+0000","last_became_active":"2024-09-13T17:42:02.340777+0000","last_became_peered":"2024-09-13T17:42:02.340777+0000","last_unstale":"2024-09-13T17:42:03.912913+0000","last_undegraded":"2024-09-13T17:42:03.912913+0000","last_fullsized":"2024-09-13T17:42:03.912913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:59:37.537177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.973354+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.969555+0000","last_became_peered":"2024-09-13T17:42:01.969555+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:14:26.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323123+0000","last_change":"2024-09-13T17:42:02.356982+0000","last_active":"2024-09-13T17:42:03.323123+0000","last_peered":"2024-09-13T17:42:03.323123+0000","last_clean":"2024-09-13T17:42:03.323123+0000","last_became_active":"2024-09-13T17:42:02.354879+0000","last_became_peered":"2024-09-13T17:42:02.354879+0000","last_unstale":"2024-09-13T17:42:03.323123+0000","last_undegraded":"2024-09-13T17:42:03.323123+0000","last_fullsized":"2024-09-13T17:42:03.323123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:51:00.269193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"14'3","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264141+0000","last_change":"2024-09-13T17:42:01.988113+0000","last_active":"2024-09-13T17:42:04.264141+0000","last_peered":"2024-09-13T17:42:04.264141+0000","last_clean":"2024-09-13T17:42:04.264141+0000","last_became_active":"2024-09-13T17:42:01.987821+0000","last_became_peered":"2024-09-13T17:42:01.987821+0000","last_unstale":"2024-09-13T17:42:04.264141+0000","last_undegraded":"2024-09-13T17:42:04.264141+0000","last_fullsized":"2024-09-13T17:42:04.264141+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263852+0000","last_change":"2024-09-13T17:42:02.342353+0000","last_active":"2024-09-13T17:42:04.263852+0000","last_peered":"2024-09-13T17:42:04.263852+0000","last_clean":"2024-09-13T17:42:04.263852+0000","last_became_active":"2024-09-13T17:42:02.336419+0000","last_became_peered":"2024-09-13T17:42:02.336419+0000","last_unstale":"2024-09-13T17:42:04.263852+0000","last_undegraded":"2024-09-13T17:42:04.263852+0000","last_fullsized":"2024-09-13T17:42:04.263852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:59:57.299409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"14'6","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912570+0000","last_change":"2024-09-13T17:42:01.981005+0000","last_active":"2024-09-13T17:42:03.912570+0000","last_peered":"2024-09-13T17:42:03.912570+0000","last_clean":"2024-09-13T17:42:03.912570+0000","last_became_active":"2024-09-13T17:42:01.978436+0000","last_became_peered":"2024-09-13T17:42:01.978436+0000","last_unstale":"2024-09-13T17:42:03.912570+0000","last_undegraded":"2024-09-13T17:42:03.912570+0000","last_fullsized":"2024-09-13T17:42:03.912570+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322389+0000","last_change":"2024-09-13T17:42:02.354827+0000","last_active":"2024-09-13T17:42:03.322389+0000","last_peered":"2024-09-13T17:42:03.322389+0000","last_clean":"2024-09-13T17:42:03.322389+0000","last_became_active":"2024-09-13T17:42:02.353803+0000","last_became_peered":"2024-09-13T17:42:02.353803+0000","last_unstale":"2024-09-13T17:42:03.322389+0000","last_undegraded":"2024-09-13T17:42:03.322389+0000","last_fullsized":"2024-09-13T17:42:03.322389+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:17.608829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322676+0000","last_change":"2024-09-13T17:42:01.981800+0000","last_active":"2024-09-13T17:42:03.322676+0000","last_peered":"2024-09-13T17:42:03.322676+0000","last_clean":"2024-09-13T17:42:03.322676+0000","last_became_active":"2024-09-13T17:42:01.978579+0000","last_became_peered":"2024-09-13T17:42:01.978579+0000","last_unstale":"2024-09-13T17:42:03.322676+0000","last_undegraded":"2024-09-13T17:42:03.322676+0000","last_fullsized":"2024-09-13T17:42:03.322676+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321641+0000","last_change":"2024-09-13T17:42:02.360084+0000","last_active":"2024-09-13T17:42:03.321641+0000","last_peered":"2024-09-13T17:42:03.321641+0000","last_clean":"2024-09-13T17:42:03.321641+0000","last_became_active":"2024-09-13T17:42:02.358943+0000","last_became_peered":"2024-09-13T17:42:02.358943+0000","last_unstale":"2024-09-13T17:42:03.321641+0000","last_undegraded":"2024-09-13T17:42:03.321641+0000","last_fullsized":"2024-09-13T17:42:03.321641+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:30.309069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911963+0000","last_change":"2024-09-13T17:42:01.980802+0000","last_active":"2024-09-13T17:42:03.911963+0000","last_peered":"2024-09-13T17:42:03.911963+0000","last_clean":"2024-09-13T17:42:03.911963+0000","last_became_active":"2024-09-13T17:42:01.977408+0000","last_became_peered":"2024-09-13T17:42:01.977408+0000","last_unstale":"2024-09-13T17:42:03.911963+0000","last_undegraded":"2024-09-13T17:42:03.911963+0000","last_fullsized":"2024-09-13T17:42:03.911963+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:59:21.671479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263263+0000","last_change":"2024-09-13T17:42:02.356627+0000","last_active":"2024-09-13T17:42:04.263263+0000","last_peered":"2024-09-13T17:42:04.263263+0000","last_clean":"2024-09-13T17:42:04.263263+0000","last_became_active":"2024-09-13T17:42:02.356381+0000","last_became_peered":"2024-09-13T17:42:02.356381+0000","last_unstale":"2024-09-13T17:42:04.263263+0000","last_undegraded":"2024-09-13T17:42:04.263263+0000","last_fullsized":"2024-09-13T17:42:04.263263+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321850+0000","last_change":"2024-09-13T17:42:01.973218+0000","last_active":"2024-09-13T17:42:03.321850+0000","last_peered":"2024-09-13T17:42:03.321850+0000","last_clean":"2024-09-13T17:42:03.321850+0000","last_became_active":"2024-09-13T17:42:01.972688+0000","last_became_peered":"2024-09-13T17:42:01.972688+0000","last_unstale":"2024-09-13T17:42:03.321850+0000","last_undegraded":"2024-09-13T17:42:03.321850+0000","last_fullsized":"2024-09-13T17:42:03.321850+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:50:27.951191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.262433+0000","last_change":"2024-09-13T17:42:02.348071+0000","last_active":"2024-09-13T17:42:04.262433+0000","last_peered":"2024-09-13T17:42:04.262433+0000","last_clean":"2024-09-13T17:42:04.262433+0000","last_became_active":"2024-09-13T17:42:02.347463+0000","last_became_peered":"2024-09-13T17:42:02.347463+0000","last_unstale":"2024-09-13T17:42:04.262433+0000","last_undegraded":"2024-09-13T17:42:04.262433+0000","last_fullsized":"2024-09-13T17:42:04.262433+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:01:44.290982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"14'8","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912464+0000","last_change":"2024-09-13T17:42:01.981250+0000","last_active":"2024-09-13T17:42:03.912464+0000","last_peered":"2024-09-13T17:42:03.912464+0000","last_clean":"2024-09-13T17:42:03.912464+0000","last_became_active":"2024-09-13T17:42:01.978674+0000","last_became_peered":"2024-09-13T17:42:01.978674+0000","last_unstale":"2024-09-13T17:42:03.912464+0000","last_undegraded":"2024-09-13T17:42:03.912464+0000","last_fullsized":"2024-09-13T17:42:03.912464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911530+0000","last_change":"2024-09-13T17:42:02.339567+0000","last_active":"2024-09-13T17:42:03.911530+0000","last_peered":"2024-09-13T17:42:03.911530+0000","last_clean":"2024-09-13T17:42:03.911530+0000","last_became_active":"2024-09-13T17:42:02.339304+0000","last_became_peered":"2024-09-13T17:42:02.339304+0000","last_unstale":"2024-09-13T17:42:03.911530+0000","last_undegraded":"2024-09-13T17:42:03.911530+0000","last_fullsized":"2024-09-13T17:42:03.911530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:05:01.291734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323607+0000","last_change":"2024-09-13T17:42:01.973755+0000","last_active":"2024-09-13T17:42:03.323607+0000","last_peered":"2024-09-13T17:42:03.323607+0000","last_clean":"2024-09-13T17:42:03.323607+0000","last_became_active":"2024-09-13T17:42:01.973179+0000","last_became_peered":"2024-09-13T17:42:01.973179+0000","last_unstale":"2024-09-13T17:42:03.323607+0000","last_undegraded":"2024-09-13T17:42:03.323607+0000","last_fullsized":"2024-09-13T17:42:03.323607+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912974+0000","last_change":"2024-09-13T17:42:02.363936+0000","last_active":"2024-09-13T17:42:03.912974+0000","last_peered":"2024-09-13T17:42:03.912974+0000","last_clean":"2024-09-13T17:42:03.912974+0000","last_became_active":"2024-09-13T17:42:02.363316+0000","last_became_peered":"2024-09-13T17:42:02.363316+0000","last_unstale":"2024-09-13T17:42:03.912974+0000","last_undegraded":"2024-09-13T17:42:03.912974+0000","last_fullsized":"2024-09-13T17:42:03.912974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:35:02.733516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913097+0000","last_change":"2024-09-13T17:42:01.987237+0000","last_active":"2024-09-13T17:42:03.913097+0000","last_peered":"2024-09-13T17:42:03.913097+0000","last_clean":"2024-09-13T17:42:03.913097+0000","last_became_active":"2024-09-13T17:42:01.986684+0000","last_became_peered":"2024-09-13T17:42:01.986684+0000","last_unstale":"2024-09-13T17:42:03.913097+0000","last_undegraded":"2024-09-13T17:42:03.913097+0000","last_fullsized":"2024-09-13T17:42:03.913097+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:27.554217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323169+0000","last_change":"2024-09-13T17:42:02.359350+0000","last_active":"2024-09-13T17:42:03.323169+0000","last_peered":"2024-09-13T17:42:03.323169+0000","last_clean":"2024-09-13T17:42:03.323169+0000","last_became_active":"2024-09-13T17:42:02.357233+0000","last_became_peered":"2024-09-13T17:42:02.357233+0000","last_unstale":"2024-09-13T17:42:03.323169+0000","last_undegraded":"2024-09-13T17:42:03.323169+0000","last_fullsized":"2024-09-13T17:42:03.323169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:24:20.541840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264444+0000","last_change":"2024-09-13T17:42:01.970689+0000","last_active":"2024-09-13T17:42:04.264444+0000","last_peered":"2024-09-13T17:42:04.264444+0000","last_clean":"2024-09-13T17:42:04.264444+0000","last_became_active":"2024-09-13T17:42:01.970057+0000","last_became_peered":"2024-09-13T17:42:01.970057+0000","last_unstale":"2024-09-13T17:42:04.264444+0000","last_undegraded":"2024-09-13T17:42:04.264444+0000","last_fullsized":"2024-09-13T17:42:04.264444+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322140+0000","last_change":"2024-09-13T17:42:02.360241+0000","last_active":"2024-09-13T17:42:03.322140+0000","last_peered":"2024-09-13T17:42:03.322140+0000","last_clean":"2024-09-13T17:42:03.322140+0000","last_became_active":"2024-09-13T17:42:02.357150+0000","last_became_peered":"2024-09-13T17:42:02.357150+0000","last_unstale":"2024-09-13T17:42:03.322140+0000","last_undegraded":"2024-09-13T17:42:03.322140+0000","last_fullsized":"2024-09-13T17:42:03.322140+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:36:55.337229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912797+0000","last_change":"2024-09-13T17:42:01.987432+0000","last_active":"2024-09-13T17:42:03.912797+0000","last_peered":"2024-09-13T17:42:03.912797+0000","last_clean":"2024-09-13T17:42:03.912797+0000","last_became_active":"2024-09-13T17:42:01.986822+0000","last_became_peered":"2024-09-13T17:42:01.986822+0000","last_unstale":"2024-09-13T17:42:03.912797+0000","last_undegraded":"2024-09-13T17:42:03.912797+0000","last_fullsized":"2024-09-13T17:42:03.912797+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:59.277624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263691+0000","last_change":"2024-09-13T17:42:02.349202+0000","last_active":"2024-09-13T17:42:04.263691+0000","last_peered":"2024-09-13T17:42:04.263691+0000","last_clean":"2024-09-13T17:42:04.263691+0000","last_became_active":"2024-09-13T17:42:02.345781+0000","last_became_peered":"2024-09-13T17:42:02.345781+0000","last_unstale":"2024-09-13T17:42:04.263691+0000","last_undegraded":"2024-09-13T17:42:04.263691+0000","last_fullsized":"2024-09-13T17:42:04.263691+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:15.393962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":141,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-13T17:42:02.317552+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:02.317552+0000","last_peered":"2024-09-13T17:42:02.317552+0000","last_clean":"2024-09-13T17:42:02.317552+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:02.317552+0000","last_undegraded":"2024-09-13T17:42:02.317552+0000","last_fullsized":"2024-09-13T17:42:02.317552+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263958+0000","last_change":"2024-09-13T17:42:01.969286+0000","last_active":"2024-09-13T17:42:04.263958+0000","last_peered":"2024-09-13T17:42:04.263958+0000","last_clean":"2024-09-13T17:42:04.263958+0000","last_became_active":"2024-09-13T17:42:01.968921+0000","last_became_peered":"2024-09-13T17:42:01.968921+0000","last_unstale":"2024-09-13T17:42:04.263958+0000","last_undegraded":"2024-09-13T17:42:04.263958+0000","last_fullsized":"2024-09-13T17:42:04.263958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263120+0000","last_change":"2024-09-13T17:42:02.329801+0000","last_active":"2024-09-13T17:42:04.263120+0000","last_peered":"2024-09-13T17:42:04.263120+0000","last_clean":"2024-09-13T17:42:04.263120+0000","last_became_active":"2024-09-13T17:42:02.329300+0000","last_became_peered":"2024-09-13T17:42:02.329300+0000","last_unstale":"2024-09-13T17:42:04.263120+0000","last_undegraded":"2024-09-13T17:42:04.263120+0000","last_fullsized":"2024-09-13T17:42:04.263120+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:52:10.081282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912706+0000","last_change":"2024-09-13T17:42:01.982284+0000","last_active":"2024-09-13T17:42:03.912706+0000","last_peered":"2024-09-13T17:42:03.912706+0000","last_clean":"2024-09-13T17:42:03.912706+0000","last_became_active":"2024-09-13T17:42:01.981495+0000","last_became_peered":"2024-09-13T17:42:01.981495+0000","last_unstale":"2024-09-13T17:42:03.912706+0000","last_undegraded":"2024-09-13T17:42:03.912706+0000","last_fullsized":"2024-09-13T17:42:03.912706+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321462+0000","last_change":"2024-09-13T17:42:02.359429+0000","last_active":"2024-09-13T17:42:03.321462+0000","last_peered":"2024-09-13T17:42:03.321462+0000","last_clean":"2024-09-13T17:42:03.321462+0000","last_became_active":"2024-09-13T17:42:02.357531+0000","last_became_peered":"2024-09-13T17:42:02.357531+0000","last_unstale":"2024-09-13T17:42:03.321462+0000","last_undegraded":"2024-09-13T17:42:03.321462+0000","last_fullsized":"2024-09-13T17:42:03.321462+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:43.826159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.911994+0000","last_change":"2024-09-13T17:42:01.980799+0000","last_active":"2024-09-13T17:42:03.911994+0000","last_peered":"2024-09-13T17:42:03.911994+0000","last_clean":"2024-09-13T17:42:03.911994+0000","last_became_active":"2024-09-13T17:42:01.977918+0000","last_became_peered":"2024-09-13T17:42:01.977918+0000","last_unstale":"2024-09-13T17:42:03.911994+0000","last_undegraded":"2024-09-13T17:42:03.911994+0000","last_fullsized":"2024-09-13T17:42:03.911994+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:48.136878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263884+0000","last_change":"2024-09-13T17:42:02.339725+0000","last_active":"2024-09-13T17:42:04.263884+0000","last_peered":"2024-09-13T17:42:04.263884+0000","last_clean":"2024-09-13T17:42:04.263884+0000","last_became_active":"2024-09-13T17:42:02.334096+0000","last_became_peered":"2024-09-13T17:42:02.334096+0000","last_unstale":"2024-09-13T17:42:04.263884+0000","last_undegraded":"2024-09-13T17:42:04.263884+0000","last_fullsized":"2024-09-13T17:42:04.263884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322118+0000","last_change":"2024-09-13T17:42:01.982743+0000","last_active":"2024-09-13T17:42:03.322118+0000","last_peered":"2024-09-13T17:42:03.322118+0000","last_clean":"2024-09-13T17:42:03.322118+0000","last_became_active":"2024-09-13T17:42:01.981897+0000","last_became_peered":"2024-09-13T17:42:01.981897+0000","last_unstale":"2024-09-13T17:42:03.322118+0000","last_undegraded":"2024-09-13T17:42:03.322118+0000","last_fullsized":"2024-09-13T17:42:03.322118+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322270+0000","last_change":"2024-09-13T17:42:02.360779+0000","last_active":"2024-09-13T17:42:03.322270+0000","last_peered":"2024-09-13T17:42:03.322270+0000","last_clean":"2024-09-13T17:42:03.322270+0000","last_became_active":"2024-09-13T17:42:02.359476+0000","last_became_peered":"2024-09-13T17:42:02.359476+0000","last_unstale":"2024-09-13T17:42:03.322270+0000","last_undegraded":"2024-09-13T17:42:03.322270+0000","last_fullsized":"2024-09-13T17:42:03.322270+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:28.876223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264113+0000","last_change":"2024-09-13T17:42:01.988534+0000","last_active":"2024-09-13T17:42:04.264113+0000","last_peered":"2024-09-13T17:42:04.264113+0000","last_clean":"2024-09-13T17:42:04.264113+0000","last_became_active":"2024-09-13T17:42:01.988254+0000","last_became_peered":"2024-09-13T17:42:01.988254+0000","last_unstale":"2024-09-13T17:42:04.264113+0000","last_undegraded":"2024-09-13T17:42:04.264113+0000","last_fullsized":"2024-09-13T17:42:04.264113+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:01:52.398009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912533+0000","last_change":"2024-09-13T17:42:02.338990+0000","last_active":"2024-09-13T17:42:03.912533+0000","last_peered":"2024-09-13T17:42:03.912533+0000","last_clean":"2024-09-13T17:42:03.912533+0000","last_became_active":"2024-09-13T17:42:02.338737+0000","last_became_peered":"2024-09-13T17:42:02.338737+0000","last_unstale":"2024-09-13T17:42:03.912533+0000","last_undegraded":"2024-09-13T17:42:03.912533+0000","last_fullsized":"2024-09-13T17:42:03.912533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:46:10.901730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264464+0000","last_change":"2024-09-13T17:42:01.966772+0000","last_active":"2024-09-13T17:42:04.264464+0000","last_peered":"2024-09-13T17:42:04.264464+0000","last_clean":"2024-09-13T17:42:04.264464+0000","last_became_active":"2024-09-13T17:42:01.965762+0000","last_became_peered":"2024-09-13T17:42:01.965762+0000","last_unstale":"2024-09-13T17:42:04.264464+0000","last_undegraded":"2024-09-13T17:42:04.264464+0000","last_fullsized":"2024-09-13T17:42:04.264464+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264436+0000","last_change":"2024-09-13T17:42:02.340638+0000","last_active":"2024-09-13T17:42:04.264436+0000","last_peered":"2024-09-13T17:42:04.264436+0000","last_clean":"2024-09-13T17:42:04.264436+0000","last_became_active":"2024-09-13T17:42:02.336028+0000","last_became_peered":"2024-09-13T17:42:02.336028+0000","last_unstale":"2024-09-13T17:42:04.264436+0000","last_undegraded":"2024-09-13T17:42:04.264436+0000","last_fullsized":"2024-09-13T17:42:04.264436+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:05.871766+0000","last_change":"2024-09-13T17:42:01.969377+0000","last_active":"2024-09-13T17:42:05.871766+0000","last_peered":"2024-09-13T17:42:05.871766+0000","last_clean":"2024-09-13T17:42:05.871766+0000","last_became_active":"2024-09-13T17:42:01.966888+0000","last_became_peered":"2024-09-13T17:42:01.966888+0000","last_unstale":"2024-09-13T17:42:05.871766+0000","last_undegraded":"2024-09-13T17:42:05.871766+0000","last_fullsized":"2024-09-13T17:42:05.871766+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:52:26.157403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912530+0000","last_change":"2024-09-13T17:42:02.336164+0000","last_active":"2024-09-13T17:42:03.912530+0000","last_peered":"2024-09-13T17:42:03.912530+0000","last_clean":"2024-09-13T17:42:03.912530+0000","last_became_active":"2024-09-13T17:42:02.335891+0000","last_became_peered":"2024-09-13T17:42:02.335891+0000","last_unstale":"2024-09-13T17:42:03.912530+0000","last_undegraded":"2024-09-13T17:42:03.912530+0000","last_fullsized":"2024-09-13T17:42:03.912530+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:09:49.230040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.264680+0000","last_change":"2024-09-13T17:42:01.974797+0000","last_active":"2024-09-13T17:42:04.264680+0000","last_peered":"2024-09-13T17:42:04.264680+0000","last_clean":"2024-09-13T17:42:04.264680+0000","last_became_active":"2024-09-13T17:42:01.971445+0000","last_became_peered":"2024-09-13T17:42:01.971445+0000","last_unstale":"2024-09-13T17:42:04.264680+0000","last_undegraded":"2024-09-13T17:42:04.264680+0000","last_fullsized":"2024-09-13T17:42:04.264680+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322893+0000","last_change":"2024-09-13T17:42:02.354060+0000","last_active":"2024-09-13T17:42:03.322893+0000","last_peered":"2024-09-13T17:42:03.322893+0000","last_clean":"2024-09-13T17:42:03.322893+0000","last_became_active":"2024-09-13T17:42:02.353776+0000","last_became_peered":"2024-09-13T17:42:02.353776+0000","last_unstale":"2024-09-13T17:42:03.322893+0000","last_undegraded":"2024-09-13T17:42:03.322893+0000","last_fullsized":"2024-09-13T17:42:03.322893+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:54.005234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322185+0000","last_change":"2024-09-13T17:42:01.983210+0000","last_active":"2024-09-13T17:42:03.322185+0000","last_peered":"2024-09-13T17:42:03.322185+0000","last_clean":"2024-09-13T17:42:03.322185+0000","last_became_active":"2024-09-13T17:42:01.978757+0000","last_became_peered":"2024-09-13T17:42:01.978757+0000","last_unstale":"2024-09-13T17:42:03.322185+0000","last_undegraded":"2024-09-13T17:42:03.322185+0000","last_fullsized":"2024-09-13T17:42:03.322185+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:13:19.013324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.321316+0000","last_change":"2024-09-13T17:42:02.359330+0000","last_active":"2024-09-13T17:42:03.321316+0000","last_peered":"2024-09-13T17:42:03.321316+0000","last_clean":"2024-09-13T17:42:03.321316+0000","last_became_active":"2024-09-13T17:42:02.358904+0000","last_became_peered":"2024-09-13T17:42:02.358904+0000","last_unstale":"2024-09-13T17:42:03.321316+0000","last_undegraded":"2024-09-13T17:42:03.321316+0000","last_fullsized":"2024-09-13T17:42:03.321316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:54:45.730743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263264+0000","last_change":"2024-09-13T17:42:01.988623+0000","last_active":"2024-09-13T17:42:04.263264+0000","last_peered":"2024-09-13T17:42:04.263264+0000","last_clean":"2024-09-13T17:42:04.263264+0000","last_became_active":"2024-09-13T17:42:01.988147+0000","last_became_peered":"2024-09-13T17:42:01.988147+0000","last_unstale":"2024-09-13T17:42:04.263264+0000","last_undegraded":"2024-09-13T17:42:04.263264+0000","last_fullsized":"2024-09-13T17:42:04.263264+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:07:14.422246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912143+0000","last_change":"2024-09-13T17:42:02.341209+0000","last_active":"2024-09-13T17:42:03.912143+0000","last_peered":"2024-09-13T17:42:03.912143+0000","last_clean":"2024-09-13T17:42:03.912143+0000","last_became_active":"2024-09-13T17:42:02.340934+0000","last_became_peered":"2024-09-13T17:42:02.340934+0000","last_unstale":"2024-09-13T17:42:03.912143+0000","last_undegraded":"2024-09-13T17:42:03.912143+0000","last_fullsized":"2024-09-13T17:42:03.912143+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:34:26.164972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263495+0000","last_change":"2024-09-13T17:42:01.988051+0000","last_active":"2024-09-13T17:42:04.263495+0000","last_peered":"2024-09-13T17:42:04.263495+0000","last_clean":"2024-09-13T17:42:04.263495+0000","last_became_active":"2024-09-13T17:42:01.987703+0000","last_became_peered":"2024-09-13T17:42:01.987703+0000","last_unstale":"2024-09-13T17:42:04.263495+0000","last_undegraded":"2024-09-13T17:42:04.263495+0000","last_fullsized":"2024-09-13T17:42:04.263495+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.288051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263566+0000","last_change":"2024-09-13T17:42:02.347875+0000","last_active":"2024-09-13T17:42:04.263566+0000","last_peered":"2024-09-13T17:42:04.263566+0000","last_clean":"2024-09-13T17:42:04.263566+0000","last_became_active":"2024-09-13T17:42:02.347616+0000","last_became_peered":"2024-09-13T17:42:02.347616+0000","last_unstale":"2024-09-13T17:42:04.263566+0000","last_undegraded":"2024-09-13T17:42:04.263566+0000","last_fullsized":"2024-09-13T17:42:04.263566+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:08.891550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322953+0000","last_change":"2024-09-13T17:42:01.988437+0000","last_active":"2024-09-13T17:42:03.322953+0000","last_peered":"2024-09-13T17:42:03.322953+0000","last_clean":"2024-09-13T17:42:03.322953+0000","last_became_active":"2024-09-13T17:42:01.987164+0000","last_became_peered":"2024-09-13T17:42:01.987164+0000","last_unstale":"2024-09-13T17:42:03.322953+0000","last_undegraded":"2024-09-13T17:42:03.322953+0000","last_fullsized":"2024-09-13T17:42:03.322953+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:42:24.082351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322005+0000","last_change":"2024-09-13T17:42:02.355026+0000","last_active":"2024-09-13T17:42:03.322005+0000","last_peered":"2024-09-13T17:42:03.322005+0000","last_clean":"2024-09-13T17:42:03.322005+0000","last_became_active":"2024-09-13T17:42:02.353935+0000","last_became_peered":"2024-09-13T17:42:02.353935+0000","last_unstale":"2024-09-13T17:42:03.322005+0000","last_undegraded":"2024-09-13T17:42:03.322005+0000","last_fullsized":"2024-09-13T17:42:03.322005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:58:54.702173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912398+0000","last_change":"2024-09-13T17:42:02.363456+0000","last_active":"2024-09-13T17:42:03.912398+0000","last_peered":"2024-09-13T17:42:03.912398+0000","last_clean":"2024-09-13T17:42:03.912398+0000","last_became_active":"2024-09-13T17:42:02.363171+0000","last_became_peered":"2024-09-13T17:42:02.363171+0000","last_unstale":"2024-09-13T17:42:03.912398+0000","last_undegraded":"2024-09-13T17:42:03.912398+0000","last_fullsized":"2024-09-13T17:42:03.912398+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:40.785623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322259+0000","last_change":"2024-09-13T17:42:01.977493+0000","last_active":"2024-09-13T17:42:03.322259+0000","last_peered":"2024-09-13T17:42:03.322259+0000","last_clean":"2024-09-13T17:42:03.322259+0000","last_became_active":"2024-09-13T17:42:01.976932+0000","last_became_peered":"2024-09-13T17:42:01.976932+0000","last_unstale":"2024-09-13T17:42:03.322259+0000","last_undegraded":"2024-09-13T17:42:03.322259+0000","last_fullsized":"2024-09-13T17:42:03.322259+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:55:21.788248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:04.263697+0000","last_change":"2024-09-13T17:42:02.348464+0000","last_active":"2024-09-13T17:42:04.263697+0000","last_peered":"2024-09-13T17:42:04.263697+0000","last_clean":"2024-09-13T17:42:04.263697+0000","last_became_active":"2024-09-13T17:42:02.348208+0000","last_became_peered":"2024-09-13T17:42:02.348208+0000","last_unstale":"2024-09-13T17:42:04.263697+0000","last_undegraded":"2024-09-13T17:42:04.263697+0000","last_fullsized":"2024-09-13T17:42:04.263697+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:15:05.160480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"14'2","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.322576+0000","last_change":"2024-09-13T17:42:01.986020+0000","last_active":"2024-09-13T17:42:03.322576+0000","last_peered":"2024-09-13T17:42:03.322576+0000","last_clean":"2024-09-13T17:42:03.322576+0000","last_became_active":"2024-09-13T17:42:01.985663+0000","last_became_peered":"2024-09-13T17:42:01.985663+0000","last_unstale":"2024-09-13T17:42:03.322576+0000","last_undegraded":"2024-09-13T17:42:03.322576+0000","last_fullsized":"2024-09-13T17:42:03.322576+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:17:12.942087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323065+0000","last_change":"2024-09-13T17:42:02.359950+0000","last_active":"2024-09-13T17:42:03.323065+0000","last_peered":"2024-09-13T17:42:03.323065+0000","last_clean":"2024-09-13T17:42:03.323065+0000","last_became_active":"2024-09-13T17:42:02.358808+0000","last_became_peered":"2024-09-13T17:42:02.358808+0000","last_unstale":"2024-09-13T17:42:03.323065+0000","last_undegraded":"2024-09-13T17:42:03.323065+0000","last_fullsized":"2024-09-13T17:42:03.323065+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:37.525653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"14'1","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.913042+0000","last_change":"2024-09-13T17:42:01.987023+0000","last_active":"2024-09-13T17:42:03.913042+0000","last_peered":"2024-09-13T17:42:03.913042+0000","last_clean":"2024-09-13T17:42:03.913042+0000","last_became_active":"2024-09-13T17:42:01.986220+0000","last_became_peered":"2024-09-13T17:42:01.986220+0000","last_unstale":"2024-09-13T17:42:03.913042+0000","last_undegraded":"2024-09-13T17:42:03.913042+0000","last_fullsized":"2024-09-13T17:42:03.913042+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:52:12.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.912685+0000","last_change":"2024-09-13T17:42:02.338763+0000","last_active":"2024-09-13T17:42:03.912685+0000","last_peered":"2024-09-13T17:42:03.912685+0000","last_clean":"2024-09-13T17:42:03.912685+0000","last_became_active":"2024-09-13T17:42:02.338494+0000","last_became_peered":"2024-09-13T17:42:02.338494+0000","last_unstale":"2024-09-13T17:42:03.912685+0000","last_undegraded":"2024-09-13T17:42:03.912685+0000","last_fullsized":"2024-09-13T17:42:03.912685+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:01.930280+0000","last_clean_scrub_stamp":"2024-09-13T17:42:01.930280+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:45:59.650608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"14'4","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-13T17:42:03.323462+0000","last_change":"2024-09-13T17:42:01.986108+0000","last_active":"2024-09-13T17:42:03.323462+0000","last_peered":"2024-09-13T17:42:03.323462+0000","last_clean":"2024-09-13T17:42:03.323462+0000","last_became_active":"2024-09-13T17:42:01.985698+0000","last_became_peered":"2024-09-13T17:42:01.985698+0000","last_unstale":"2024-09-13T17:42:03.323462+0000","last_undegraded":"2024-09-13T17:42:03.323462+0000","last_fullsized":"2024-09-13T17:42:03.323462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:00.918750+0000","last_clean_scrub_stamp":"2024-09-13T17:42:00.918750+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:07.076785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738375,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":151552,"data_stored":42001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":753664,"data_stored":633029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":741376,"data_stored":631429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:13.563 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:42:13.563 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T17:42:13.563 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T17:42:13.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T17:42:13.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e44821640 1 Processor -- start 2024-09-13T17:42:13.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e44821640 1 -- start start 2024-09-13T17:42:13.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e44821640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e4009e820 0x7f0e4013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e44821640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e400a68b0 con 0x7f0e40134e70 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e44821640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4009ddc0 con 0x7f0e4009e820 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e3e59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0e40134e70 0x7f0e40135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58952/0 (socket says 172.21.15.96:58952) 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e3e59c640 1 -- 172.21.15.96:0/1187783910 learned_addr learned my addr 172.21.15.96:0/1187783910 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.717+0000 7f0e3dd9b640 1 --2- 172.21.15.96:0/1187783910 >> v2:172.21.15.96:3300/0 conn(0x7f0e4009e820 0x7f0e4013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3d59a640 1 -- 172.21.15.96:0/1187783910 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472600163 0 0) 0x7f0e400a68b0 con 0x7f0e40134e70 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3d59a640 1 -- 172.21.15.96:0/1187783910 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e28003530 con 0x7f0e40134e70 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3dd9b640 1 -- 172.21.15.96:0/1187783910 >> v1:172.21.15.96:6789/0 conn(0x7f0e40134e70 legacy=0x7f0e40135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3dd9b640 1 -- 172.21.15.96:0/1187783910 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e40136970 con 0x7f0e4009e820 2024-09-13T17:42:13.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3dd9b640 1 --2- 172.21.15.96:0/1187783910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4013ae50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009740 tx=0x7f0e3401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6403fc4e7344c907 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3d59a640 1 -- 172.21.15.96:0/1187783910 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e340ab530 con 0x7f0e4009e820 2024-09-13T17:42:13.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3d59a640 1 -- 172.21.15.96:0/1187783910 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e340a7050 con 0x7f0e4009e820 2024-09-13T17:42:13.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e3d59a640 1 -- 172.21.15.96:0/1187783910 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e340ab7e0 con 0x7f0e4009e820 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 -- 172.21.15.96:0/1187783910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 msgr2=0x7f0e4013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 --2- 172.21.15.96:0/1187783910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4013ae50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009740 tx=0x7f0e3401c420 comp rx=0 tx=0).stop 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 -- 172.21.15.96:0/1187783910 shutdown_connections 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 --2- 172.21.15.96:0/1187783910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4013ae50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 -- 172.21.15.96:0/1187783910 >> 172.21.15.96:0/1187783910 conn(0x7f0e40098a60 msgr2=0x7f0e4009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 -- 172.21.15.96:0/1187783910 shutdown_connections 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.721+0000 7f0e44821640 1 -- 172.21.15.96:0/1187783910 wait complete. 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 Processor -- start 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 -- start start 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e400a25a0 con 0x7f0e4009e820 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39448/0 (socket says 172.21.15.96:39448) 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3e59c640 1 -- 172.21.15.96:0/1030780162 learned_addr learned my addr 172.21.15.96:0/1030780162 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3e59c640 1 -- 172.21.15.96:0/1030780162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e34009420 con 0x7f0e4009e820 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3e59c640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000cb40 tx=0x7f0e300021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e3000ab80 con 0x7f0e4009e820 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3000ace0 con 0x7f0e4009e820 2024-09-13T17:42:13.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e400a3240 con 0x7f0e4009e820 2024-09-13T17:42:13.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e30012400 con 0x7f0e4009e820 2024-09-13T17:42:13.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4012fb80 con 0x7f0e4009e820 2024-09-13T17:42:13.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0e30007070 con 0x7f0e4009e820 2024-09-13T17:42:13.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e400a0710 con 0x7f0e4009e820 2024-09-13T17:42:13.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 0x7f0e20037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:13.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f0e300476f0 con 0x7f0e4009e820 2024-09-13T17:42:13.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3dd9b640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 0x7f0e20037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:13.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.725+0000 7f0e3dd9b640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 0x7f0e20037910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ad000 tx=0x7f0e34008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:13.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.729+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e300198b0 con 0x7f0e4009e820 2024-09-13T17:42:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0e400a2a30 con 0x7f0e4009e820 2024-09-13T17:42:13.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e2effd640 1 -- 172.21.15.96:0/1030780162 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0e300192c0 con 0x7f0e4009e820 2024-09-13T17:42:13.909 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:13.909 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 msgr2=0x7f0e20037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 0x7f0e20037910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e340ad000 tx=0x7f0e34008040 comp rx=0 tx=0).stop 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 msgr2=0x7f0e4009f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0e3000cb40 tx=0x7f0e300021f0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 shutdown_connections 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e20035450 0x7f0e20037910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 --2- 172.21.15.96:0/1030780162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4009e820 0x7f0e4009f280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:13.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 >> 172.21.15.96:0/1030780162 conn(0x7f0e40098a60 msgr2=0x7f0e40132510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:13.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 shutdown_connections 2024-09-13T17:42:13.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:13.909+0000 7f0e44821640 1 -- 172.21.15.96:0/1030780162 wait complete. 2024-09-13T17:42:13.925 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T17:42:13.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:14.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86bb027640 1 Processor -- start 2024-09-13T17:42:14.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86bb027640 1 -- start start 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86bb027640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f86b409dc20 0x7f86b41351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86bb027640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86b406e130 con 0x7f86b406f6d0 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86bb027640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86b40a6750 con 0x7f86b409dc20 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86abfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f86b409dc20 0x7f86b41351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86b8dc3640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f86b406f6d0 0x7f86b4134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58966/0 (socket says 172.21.15.96:58966) 2024-09-13T17:42:14.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86b8dc3640 1 -- 172.21.15.96:0/3354221557 learned_addr learned my addr 172.21.15.96:0/3354221557 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86abfff640 1 -- 172.21.15.96:0/3354221557 >> v1:172.21.15.96:6789/0 conn(0x7f86b406f6d0 legacy=0x7f86b4134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86abfff640 1 -- 172.21.15.96:0/3354221557 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b40712e0 con 0x7f86b409dc20 2024-09-13T17:42:14.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.049+0000 7f86ab7fe640 1 -- 172.21.15.96:0/3354221557 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566260375 0 0) 0x7f86b406e130 con 0x7f86b406f6d0 2024-09-13T17:42:14.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86abfff640 1 --2- 172.21.15.96:0/3354221557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b409dc20 0x7f86b41351c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009740 tx=0x7f86a401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddddaf2e92c6600 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86ab7fe640 1 -- 172.21.15.96:0/3354221557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86a40ab400 con 0x7f86b409dc20 2024-09-13T17:42:14.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86ab7fe640 1 -- 172.21.15.96:0/3354221557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a40a7050 con 0x7f86b409dc20 2024-09-13T17:42:14.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86ab7fe640 1 -- 172.21.15.96:0/3354221557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86a40ab6b0 con 0x7f86b409dc20 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- 172.21.15.96:0/3354221557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b409dc20 msgr2=0x7f86b41351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 --2- 172.21.15.96:0/3354221557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b409dc20 0x7f86b41351c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86a4009740 tx=0x7f86a401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- 172.21.15.96:0/3354221557 shutdown_connections 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 --2- 172.21.15.96:0/3354221557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b409dc20 0x7f86b41351c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- 172.21.15.96:0/3354221557 >> 172.21.15.96:0/3354221557 conn(0x7f86b4098a60 msgr2=0x7f86b409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- 172.21.15.96:0/3354221557 shutdown_connections 2024-09-13T17:42:14.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- 172.21.15.96:0/3354221557 wait complete. 2024-09-13T17:42:14.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 Processor -- start 2024-09-13T17:42:14.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- start start 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86bb027640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b41c8aa0 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86b8dc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86b8dc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39472/0 (socket says 172.21.15.96:39472) 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86b8dc3640 1 -- 172.21.15.96:0/1569370933 learned_addr learned my addr 172.21.15.96:0/1569370933 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.053+0000 7f86b8dc3640 1 -- 172.21.15.96:0/1569370933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a4009420 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86b8dc3640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c420 tx=0x7f86b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b00074d0 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b0007630 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b40a02b0 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f86b0004960 con 0x7f86b406f6d0 2024-09-13T17:42:14.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86b40a1b70 con 0x7f86b406f6d0 2024-09-13T17:42:14.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f86b0004ac0 con 0x7f86b406f6d0 2024-09-13T17:42:14.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86b409ff60 con 0x7f86b406f6d0 2024-09-13T17:42:14.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 0x7f86a0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f86b0046c60 con 0x7f86b406f6d0 2024-09-13T17:42:14.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86abfff640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 0x7f86a0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.057+0000 7f86abfff640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 0x7f86a0037910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ad000 tx=0x7f86a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.061+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f86b001acd0 con 0x7f86b406f6d0 2024-09-13T17:42:14.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86b4134ab0 con 0x7f86b406f6d0 2024-09-13T17:42:14.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86a97fa640 1 -- 172.21.15.96:0/1569370933 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f86b0018020 con 0x7f86b406f6d0 2024-09-13T17:42:14.217 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:14.217 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 msgr2=0x7f86a0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 0x7f86a0037910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f86a40ad000 tx=0x7f86a4008040 comp rx=0 tx=0).stop 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 msgr2=0x7f86b40a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f86b000c420 tx=0x7f86b000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 shutdown_connections 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f86a0035450 0x7f86a0037910 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 --2- 172.21.15.96:0/1569370933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86b406f6d0 0x7f86b40a0db0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 >> 172.21.15.96:0/1569370933 conn(0x7f86b4098a60 msgr2=0x7f86b40702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 shutdown_connections 2024-09-13T17:42:14.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.217+0000 7f86bb027640 1 -- 172.21.15.96:0/1569370933 wait complete. 2024-09-13T17:42:14.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:14.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:14.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.353+0000 7ff998388640 1 Processor -- start 2024-09-13T17:42:14.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.353+0000 7ff998388640 1 -- start start 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.353+0000 7ff998388640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff99009e820 0x7ff99013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.353+0000 7ff998388640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900a68b0 con 0x7ff990134e70 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.353+0000 7ff998388640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99009ddc0 con 0x7ff99009e820 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff99009e820 0x7ff99013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff99009e820 0x7ff99013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39482/0 (socket says 172.21.15.96:39482) 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 -- 172.21.15.96:0/771606743 learned_addr learned my addr 172.21.15.96:0/771606743 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff996124640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff990134e70 0x7ff990135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58982/0 (socket says 172.21.15.96:58982) 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 -- 172.21.15.96:0/771606743 >> v1:172.21.15.96:6789/0 conn(0x7ff990134e70 legacy=0x7ff990135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 -- 172.21.15.96:0/771606743 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff990136970 con 0x7ff99009e820 2024-09-13T17:42:14.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995923640 1 --2- 172.21.15.96:0/771606743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99013ae50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097f0 tx=0x7ff98001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8841e6851caf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995122640 1 -- 172.21.15.96:0/771606743 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9800ac540 con 0x7ff99009e820 2024-09-13T17:42:14.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995122640 1 -- 172.21.15.96:0/771606743 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9800ac6a0 con 0x7ff99009e820 2024-09-13T17:42:14.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff995122640 1 -- 172.21.15.96:0/771606743 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9800ac960 con 0x7ff99009e820 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 -- 172.21.15.96:0/771606743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 msgr2=0x7ff99013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 --2- 172.21.15.96:0/771606743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99013ae50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff9800097f0 tx=0x7ff98001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 -- 172.21.15.96:0/771606743 shutdown_connections 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 --2- 172.21.15.96:0/771606743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99013ae50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 -- 172.21.15.96:0/771606743 >> 172.21.15.96:0/771606743 conn(0x7ff990098a60 msgr2=0x7ff99009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 -- 172.21.15.96:0/771606743 shutdown_connections 2024-09-13T17:42:14.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.357+0000 7ff998388640 1 -- 172.21.15.96:0/771606743 wait complete. 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 Processor -- start 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 -- start start 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900a3ff0 con 0x7ff99009e820 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff996124640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff996124640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39492/0 (socket says 172.21.15.96:39492) 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff996124640 1 -- 172.21.15.96:0/254999227 learned_addr learned my addr 172.21.15.96:0/254999227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff996124640 1 -- 172.21.15.96:0/254999227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9800094d0 con 0x7ff99009e820 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff996124640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c3a0 tx=0x7ff98c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff98c007410 con 0x7ff99009e820 2024-09-13T17:42:14.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98c007570 con 0x7ff99009e820 2024-09-13T17:42:14.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff98c0047f0 con 0x7ff99009e820 2024-09-13T17:42:14.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9900a4340 con 0x7ff99009e820 2024-09-13T17:42:14.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99013aba0 con 0x7ff99009e820 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff98c004a60 con 0x7ff99009e820 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff99013a650 con 0x7ff99009e820 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 0x7ff9740378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff98c047160 con 0x7ff99009e820 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff995923640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 0x7ff9740378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.361+0000 7ff995923640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 0x7ff9740378c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ae000 tx=0x7ff980008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.365+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff98c0196d0 con 0x7ff99009e820 2024-09-13T17:42:14.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.513+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff99006a1b0 con 0x7ff99009e820 2024-09-13T17:42:14.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.513+0000 7ff986ffd640 1 -- 172.21.15.96:0/254999227 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff98c0113a0 con 0x7ff99009e820 2024-09-13T17:42:14.515 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 msgr2=0x7ff9740378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 0x7ff9740378c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff9800ae000 tx=0x7ff980008040 comp rx=0 tx=0).stop 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 msgr2=0x7ff99006f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c3a0 tx=0x7ff98c00c860 comp rx=0 tx=0).stop 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 shutdown_connections 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff974035400 0x7ff9740378c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 --2- 172.21.15.96:0/254999227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99009e820 0x7ff99006f750 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 >> 172.21.15.96:0/254999227 conn(0x7ff990098a60 msgr2=0x7ff990134070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 shutdown_connections 2024-09-13T17:42:14.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.517+0000 7ff998388640 1 -- 172.21.15.96:0/254999227 wait complete. 2024-09-13T17:42:14.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:14.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb75517d640 1 Processor -- start 2024-09-13T17:42:14.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb75517d640 1 -- start start 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb75517d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb750134e70 0x7fb75009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb75517d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75009ed60 con 0x7fb750135250 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb75517d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75009ddc0 con 0x7fb750134e70 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74ed9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb750134e70 0x7fb75009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb747fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb750135250 0x7fb750135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:58998/0 (socket says 172.21.15.96:58998) 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb747fff640 1 -- 172.21.15.96:0/982033900 learned_addr learned my addr 172.21.15.96:0/982033900 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74ed9d640 1 -- 172.21.15.96:0/982033900 >> v1:172.21.15.96:6789/0 conn(0x7fb750135250 legacy=0x7fb750135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74ed9d640 1 -- 172.21.15.96:0/982033900 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7500a68b0 con 0x7fb750134e70 2024-09-13T17:42:14.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74ed9d640 1 --2- 172.21.15.96:0/982033900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75009e820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb740009740 tx=0x7fb74001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d750a3be811d3dea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74e59c640 1 -- 172.21.15.96:0/982033900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7400ab530 con 0x7fb750134e70 2024-09-13T17:42:14.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74e59c640 1 -- 172.21.15.96:0/982033900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400a7050 con 0x7fb750134e70 2024-09-13T17:42:14.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.645+0000 7fb74e59c640 1 -- 172.21.15.96:0/982033900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7400ab7e0 con 0x7fb750134e70 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/982033900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 msgr2=0x7fb75009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 --2- 172.21.15.96:0/982033900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75009e820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb740009740 tx=0x7fb74001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/982033900 shutdown_connections 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 --2- 172.21.15.96:0/982033900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75009e820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/982033900 >> 172.21.15.96:0/982033900 conn(0x7fb750098a60 msgr2=0x7fb75009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/982033900 shutdown_connections 2024-09-13T17:42:14.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/982033900 wait complete. 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 Processor -- start 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- start start 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb750070990 con 0x7fb750134e70 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb74ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39498/0 (socket says 172.21.15.96:39498) 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb74ed9d640 1 -- 172.21.15.96:0/2898666426 learned_addr learned my addr 172.21.15.96:0/2898666426 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb74ed9d640 1 -- 172.21.15.96:0/2898666426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740009420 con 0x7fb750134e70 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb74ed9d640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb74001a050 tx=0x7fb7400191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7400abea0 con 0x7fb750134e70 2024-09-13T17:42:14.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7400b4410 con 0x7fb750134e70 2024-09-13T17:42:14.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7500a45f0 con 0x7fb750134e70 2024-09-13T17:42:14.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb7400b46b0 con 0x7fb750134e70 2024-09-13T17:42:14.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7500a57a0 con 0x7fb750134e70 2024-09-13T17:42:14.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb7400b2040 con 0x7fb750134e70 2024-09-13T17:42:14.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7500a4d40 con 0x7fb750134e70 2024-09-13T17:42:14.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 0x7fb730037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb7400e18d0 con 0x7fb750134e70 2024-09-13T17:42:14.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.649+0000 7fb747fff640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 0x7fb730037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.653+0000 7fb747fff640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 0x7fb730037910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb738001610 tx=0x7fb738007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.657+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb7400bc3b0 con 0x7fb750134e70 2024-09-13T17:42:14.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.809+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb750070ce0 con 0x7fb750134e70 2024-09-13T17:42:14.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb7477fe640 1 -- 172.21.15.96:0/2898666426 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7400b0070 con 0x7fb750134e70 2024-09-13T17:42:14.812 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 msgr2=0x7fb730037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 0x7fb730037910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb738001610 tx=0x7fb738007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 msgr2=0x7fb75013b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb74001a050 tx=0x7fb7400191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 shutdown_connections 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb730035450 0x7fb730037910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 --2- 172.21.15.96:0/2898666426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb750134e70 0x7fb75013b3d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 >> 172.21.15.96:0/2898666426 conn(0x7fb750098a60 msgr2=0x7fb7500995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 shutdown_connections 2024-09-13T17:42:14.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.813+0000 7fb75517d640 1 -- 172.21.15.96:0/2898666426 wait complete. 2024-09-13T17:42:14.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:14.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.953+0000 7f65028f9640 1 Processor -- start 2024-09-13T17:42:14.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.953+0000 7f65028f9640 1 -- start start 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.953+0000 7f65028f9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f64fc134e70 0x7f64fc09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.953+0000 7f65028f9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc09ed60 con 0x7f64fc135250 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.953+0000 7f65028f9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc09ddc0 con 0x7f64fc134e70 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f64fc134e70 0x7f64fc09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f64fc134e70 0x7f64fc09e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39514/0 (socket says 172.21.15.96:39514) 2024-09-13T17:42:14.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fb7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f64fc135250 0x7f64fc135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59010/0 (socket says 172.21.15.96:59010) 2024-09-13T17:42:14.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 -- 172.21.15.96:0/2788694089 learned_addr learned my addr 172.21.15.96:0/2788694089 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 -- 172.21.15.96:0/2788694089 >> v1:172.21.15.96:6789/0 conn(0x7f64fc135250 legacy=0x7f64fc135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 -- 172.21.15.96:0/2788694089 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0a68b0 con 0x7f64fc134e70 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64fbfff640 1 --2- 172.21.15.96:0/2788694089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09e820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097f0 tx=0x7f64e401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e728dce8baf62ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64faffd640 1 -- 172.21.15.96:0/2788694089 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64e40ac540 con 0x7f64fc134e70 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64faffd640 1 -- 172.21.15.96:0/2788694089 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e40ac6a0 con 0x7f64fc134e70 2024-09-13T17:42:14.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f64faffd640 1 -- 172.21.15.96:0/2788694089 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64e40ac960 con 0x7f64fc134e70 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 -- 172.21.15.96:0/2788694089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 msgr2=0x7f64fc09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 --2- 172.21.15.96:0/2788694089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09e820 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f64e40097f0 tx=0x7f64e401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 -- 172.21.15.96:0/2788694089 shutdown_connections 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 --2- 172.21.15.96:0/2788694089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09e820 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 -- 172.21.15.96:0/2788694089 >> 172.21.15.96:0/2788694089 conn(0x7f64fc098a60 msgr2=0x7f64fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 -- 172.21.15.96:0/2788694089 shutdown_connections 2024-09-13T17:42:14.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.957+0000 7f65028f9640 1 -- 172.21.15.96:0/2788694089 wait complete. 2024-09-13T17:42:14.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 Processor -- start 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 -- start start 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0a5e30 con 0x7f64fc134e70 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39516/0 (socket says 172.21.15.96:39516) 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fbfff640 1 -- 172.21.15.96:0/2289596481 learned_addr learned my addr 172.21.15.96:0/2289596481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fbfff640 1 -- 172.21.15.96:0/2289596481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40094d0 con 0x7f64fc134e70 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fbfff640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64e40190f0 tx=0x7f64e4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64e40bb070 con 0x7f64fc134e70 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e40b7470 con 0x7f64fc134e70 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64e40b7730 con 0x7f64fc134e70 2024-09-13T17:42:14.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc0a23f0 con 0x7f64fc134e70 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc0a0890 con 0x7f64fc134e70 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f64e40b3070 con 0x7f64fc134e70 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 0x7f64dc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f64e40e17a0 con 0x7f64fc134e70 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0a20e0 con 0x7f64fc134e70 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fb7fe640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 0x7f64dc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:14.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.961+0000 7f64fb7fe640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 0x7f64dc037910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001600 tx=0x7f64ec007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:14.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:14.965+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64e40b1070 con 0x7f64fc134e70 2024-09-13T17:42:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64fc1368b0 con 0x7f64fc134e70 2024-09-13T17:42:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f64f8ff9640 1 -- 172.21.15.96:0/2289596481 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64e40abd40 con 0x7f64fc134e70 2024-09-13T17:42:15.113 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 msgr2=0x7f64dc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 0x7f64dc037910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64ec001600 tx=0x7f64ec007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 msgr2=0x7f64fc09f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f64e40190f0 tx=0x7f64e4019280 comp rx=0 tx=0).stop 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 shutdown_connections 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f64dc035450 0x7f64dc037910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 --2- 172.21.15.96:0/2289596481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f64fc134e70 0x7f64fc09f280 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 >> 172.21.15.96:0/2289596481 conn(0x7f64fc098a60 msgr2=0x7f64fc0995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 shutdown_connections 2024-09-13T17:42:15.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.113+0000 7f65028f9640 1 -- 172.21.15.96:0/2289596481 wait complete. 2024-09-13T17:42:15.130 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:15.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.237+0000 7f97edf10640 1 Processor -- start 2024-09-13T17:42:15.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.237+0000 7f97edf10640 1 -- start start 2024-09-13T17:42:15.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97e809dc20 0x7f97e81351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e806e130 con 0x7f97e806f6d0 2024-09-13T17:42:15.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e80a6750 con 0x7f97e809dc20 2024-09-13T17:42:15.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97e809dc20 0x7f97e81351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e77fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f97e806f6d0 0x7f97e8134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59022/0 (socket says 172.21.15.96:59022) 2024-09-13T17:42:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e77fe640 1 -- 172.21.15.96:0/3446717967 learned_addr learned my addr 172.21.15.96:0/3446717967 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e6ffd640 1 -- 172.21.15.96:0/3446717967 >> v1:172.21.15.96:6789/0 conn(0x7f97e806f6d0 legacy=0x7f97e8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e67fc640 1 -- 172.21.15.96:0/3446717967 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791848943 0 0) 0x7f97e806e130 con 0x7f97e806f6d0 2024-09-13T17:42:15.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e6ffd640 1 -- 172.21.15.96:0/3446717967 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80712e0 con 0x7f97e809dc20 2024-09-13T17:42:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e6ffd640 1 --2- 172.21.15.96:0/3446717967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e809dc20 0x7f97e81351c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009740 tx=0x7f97d001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f2f0fb018a6d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e67fc640 1 -- 172.21.15.96:0/3446717967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97d00ab400 con 0x7f97e809dc20 2024-09-13T17:42:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e67fc640 1 -- 172.21.15.96:0/3446717967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d00a7050 con 0x7f97e809dc20 2024-09-13T17:42:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97e67fc640 1 -- 172.21.15.96:0/3446717967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97d00ab6b0 con 0x7f97e809dc20 2024-09-13T17:42:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- 172.21.15.96:0/3446717967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e809dc20 msgr2=0x7f97e81351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 --2- 172.21.15.96:0/3446717967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e809dc20 0x7f97e81351c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f97d0009740 tx=0x7f97d001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- 172.21.15.96:0/3446717967 shutdown_connections 2024-09-13T17:42:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 --2- 172.21.15.96:0/3446717967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e809dc20 0x7f97e81351c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- 172.21.15.96:0/3446717967 >> 172.21.15.96:0/3446717967 conn(0x7f97e8098a60 msgr2=0x7f97e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- 172.21.15.96:0/3446717967 shutdown_connections 2024-09-13T17:42:15.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.241+0000 7f97edf10640 1 -- 172.21.15.96:0/3446717967 wait complete. 2024-09-13T17:42:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 Processor -- start 2024-09-13T17:42:15.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 -- start start 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8121800 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39528/0 (socket says 172.21.15.96:39528) 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e77fe640 1 -- 172.21.15.96:0/3343714989 learned_addr learned my addr 172.21.15.96:0/3343714989 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e77fe640 1 -- 172.21.15.96:0/3343714989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009420 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e77fe640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c840 tx=0x7f97e000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97e000a660 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e000a7e0 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e8120880 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97e000aac0 con 0x7f97e806f6d0 2024-09-13T17:42:15.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e80a5aa0 con 0x7f97e806f6d0 2024-09-13T17:42:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f97e000ac20 con 0x7f97e806f6d0 2024-09-13T17:42:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e80a55a0 con 0x7f97e806f6d0 2024-09-13T17:42:15.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 0x7f97c8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f97e0046fe0 con 0x7f97e806f6d0 2024-09-13T17:42:15.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e6ffd640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 0x7f97c8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.245+0000 7f97e6ffd640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 0x7f97c8037910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ad000 tx=0x7f97d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.249+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97e0078db0 con 0x7f97e806f6d0 2024-09-13T17:42:15.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.429+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f97e8134a80 con 0x7f97e806f6d0 2024-09-13T17:42:15.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97dffff640 1 -- 172.21.15.96:0/3343714989 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f97e001a6b0 con 0x7f97e806f6d0 2024-09-13T17:42:15.432 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:15.432 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:15.432 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 msgr2=0x7f97c8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 0x7f97c8037910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f97d00ad000 tx=0x7f97d0008040 comp rx=0 tx=0).stop 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 msgr2=0x7f97e811ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c840 tx=0x7f97e000cd00 comp rx=0 tx=0).stop 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 shutdown_connections 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97c8035450 0x7f97c8037910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 --2- 172.21.15.96:0/3343714989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e806f6d0 0x7f97e811ce70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 >> 172.21.15.96:0/3343714989 conn(0x7f97e8098a60 msgr2=0x7f97e8139550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 shutdown_connections 2024-09-13T17:42:15.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.433+0000 7f97edf10640 1 -- 172.21.15.96:0/3343714989 wait complete. 2024-09-13T17:42:15.449 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f005230c640 1 Processor -- start 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f005230c640 1 -- start start 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f005230c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f004c09e820 0x7f004c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f005230c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0a68b0 con 0x7f004c134e70 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f005230c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f004c09ddc0 con 0x7f004c09e820 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f004c134e70 0x7f004c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59026/0 (socket says 172.21.15.96:59026) 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004bfff640 1 -- 172.21.15.96:0/4275338429 learned_addr learned my addr 172.21.15.96:0/4275338429 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004b7fe640 1 --2- 172.21.15.96:0/4275338429 >> v2:172.21.15.96:3300/0 conn(0x7f004c09e820 0x7f004c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004b7fe640 1 -- 172.21.15.96:0/4275338429 >> v1:172.21.15.96:6789/0 conn(0x7f004c134e70 legacy=0x7f004c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004b7fe640 1 -- 172.21.15.96:0/4275338429 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c136970 con 0x7f004c09e820 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.561+0000 7f004affd640 1 -- 172.21.15.96:0/4275338429 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884767402 0 0) 0x7f004c0a68b0 con 0x7f004c134e70 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004b7fe640 1 --2- 172.21.15.96:0/4275338429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c13ae50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0040009740 tx=0x7f004001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92965ead51ec888f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004affd640 1 -- 172.21.15.96:0/4275338429 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00400ab530 con 0x7f004c09e820 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004affd640 1 -- 172.21.15.96:0/4275338429 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00400a7050 con 0x7f004c09e820 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004affd640 1 -- 172.21.15.96:0/4275338429 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00400ab7e0 con 0x7f004c09e820 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- 172.21.15.96:0/4275338429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 msgr2=0x7f004c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 --2- 172.21.15.96:0/4275338429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c13ae50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0040009740 tx=0x7f004001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- 172.21.15.96:0/4275338429 shutdown_connections 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 --2- 172.21.15.96:0/4275338429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c13ae50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- 172.21.15.96:0/4275338429 >> 172.21.15.96:0/4275338429 conn(0x7f004c098a60 msgr2=0x7f004c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- 172.21.15.96:0/4275338429 shutdown_connections 2024-09-13T17:42:15.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- 172.21.15.96:0/4275338429 wait complete. 2024-09-13T17:42:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 Processor -- start 2024-09-13T17:42:15.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- start start 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f005230c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0a1200 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39542/0 (socket says 172.21.15.96:39542) 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.565+0000 7f004bfff640 1 -- 172.21.15.96:0/150513530 learned_addr learned my addr 172.21.15.96:0/150513530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f004bfff640 1 -- 172.21.15.96:0/150513530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0040009420 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f004bfff640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f004400c840 tx=0x7f004400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f004400a660 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004400a7e0 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c0a6150 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f004400aac0 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c0a1900 con 0x7f004c09e820 2024-09-13T17:42:15.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f004400ac20 con 0x7f004c09e820 2024-09-13T17:42:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c0a0a80 con 0x7f004c09e820 2024-09-13T17:42:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 0x7f0038037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f0044047070 con 0x7f004c09e820 2024-09-13T17:42:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f004b7fe640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 0x7f0038037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.569+0000 7f004b7fe640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 0x7f0038037910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00400ad000 tx=0x7f0040008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.573+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0044078db0 con 0x7f004c09e820 2024-09-13T17:42:15.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.721+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f004c0a62e0 con 0x7f004c09e820 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.721+0000 7f0048ff9640 1 -- 172.21.15.96:0/150513530 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f0044018030 con 0x7f004c09e820 2024-09-13T17:42:15.722 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:15.723 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 msgr2=0x7f0038037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 0x7f0038037910 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f00400ad000 tx=0x7f0040008040 comp rx=0 tx=0).stop 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 msgr2=0x7f004c09f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f004400c840 tx=0x7f004400cd00 comp rx=0 tx=0).stop 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 shutdown_connections 2024-09-13T17:42:15.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0038035450 0x7f0038037910 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 --2- 172.21.15.96:0/150513530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f004c09e820 0x7f004c09f280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 >> 172.21.15.96:0/150513530 conn(0x7f004c098a60 msgr2=0x7f004c06dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 shutdown_connections 2024-09-13T17:42:15.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.725+0000 7f005230c640 1 -- 172.21.15.96:0/150513530 wait complete. 2024-09-13T17:42:15.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:15.741 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:15.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b661e2640 1 Processor -- start 2024-09-13T17:42:15.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b661e2640 1 -- start start 2024-09-13T17:42:15.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b661e2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5b6009dc20 0x7f5b601351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b661e2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b6006e130 con 0x7f5b6006f6d0 2024-09-13T17:42:15.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b661e2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b600a6750 con 0x7f5b6009dc20 2024-09-13T17:42:15.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5b6009dc20 0x7f5b601351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5b6009dc20 0x7f5b601351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39546/0 (socket says 172.21.15.96:39546) 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5effd640 1 -- 172.21.15.96:0/3182838120 learned_addr learned my addr 172.21.15.96:0/3182838120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5f7fe640 1 --1- 172.21.15.96:0/3182838120 >> v1:172.21.15.96:6789/0 conn(0x7f5b6006f6d0 0x7f5b60134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59032/0 (socket says 172.21.15.96:59032) 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5effd640 1 -- 172.21.15.96:0/3182838120 >> v1:172.21.15.96:6789/0 conn(0x7f5b6006f6d0 legacy=0x7f5b60134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5effd640 1 -- 172.21.15.96:0/3182838120 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b600712e0 con 0x7f5b6009dc20 2024-09-13T17:42:15.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.857+0000 7f5b5e7fc640 1 -- 172.21.15.96:0/3182838120 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513076278 0 0) 0x7f5b6006e130 con 0x7f5b6006f6d0 2024-09-13T17:42:15.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5effd640 1 --2- 172.21.15.96:0/3182838120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6009dc20 0x7f5b601351c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009e50 tx=0x7f5b5001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7636ee62483be7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5e7fc640 1 -- 172.21.15.96:0/3182838120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b500090d0 con 0x7f5b6009dc20 2024-09-13T17:42:15.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5e7fc640 1 -- 172.21.15.96:0/3182838120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b50009230 con 0x7f5b6009dc20 2024-09-13T17:42:15.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5e7fc640 1 -- 172.21.15.96:0/3182838120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b500a5750 con 0x7f5b6009dc20 2024-09-13T17:42:15.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/3182838120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6009dc20 msgr2=0x7f5b601351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:15.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 --2- 172.21.15.96:0/3182838120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6009dc20 0x7f5b601351c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009e50 tx=0x7f5b5001c9a0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/3182838120 shutdown_connections 2024-09-13T17:42:15.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 --2- 172.21.15.96:0/3182838120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6009dc20 0x7f5b601351c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:15.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/3182838120 >> 172.21.15.96:0/3182838120 conn(0x7f5b60098a60 msgr2=0x7f5b6009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:15.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/3182838120 shutdown_connections 2024-09-13T17:42:15.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/3182838120 wait complete. 2024-09-13T17:42:15.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 Processor -- start 2024-09-13T17:42:15.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- start start 2024-09-13T17:42:15.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b601210a0 con 0x7f5b6006f6d0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39548/0 (socket says 172.21.15.96:39548) 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5f7fe640 1 -- 172.21.15.96:0/1912157815 learned_addr learned my addr 172.21.15.96:0/1912157815 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5f7fe640 1 -- 172.21.15.96:0/1912157815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b50009b30 con 0x7f5b6006f6d0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b5f7fe640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c420 tx=0x7f5b5800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b58006470 con 0x7f5b6006f6d0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580065d0 con 0x7f5b6006f6d0 2024-09-13T17:42:15.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b601220f0 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b58004940 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.861+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b600a59f0 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5b58004aa0 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6009ac90 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b47fff640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 0x7f5b4c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5b58047190 con 0x7f5b6006f6d0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b5effd640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 0x7f5b4c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:15.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.865+0000 7f5b5effd640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 0x7f5b4c037910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b500ae000 tx=0x7f5b500a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:15.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:15.869+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b58004d50 con 0x7f5b6006f6d0 2024-09-13T17:42:16.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.013+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b6006f310 con 0x7f5b6006f6d0 2024-09-13T17:42:16.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.013+0000 7f5b47fff640 1 -- 172.21.15.96:0/1912157815 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b5801a360 con 0x7f5b6006f6d0 2024-09-13T17:42:16.014 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:16.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 msgr2=0x7f5b4c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 0x7f5b4c037910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5b500ae000 tx=0x7f5b500a7070 comp rx=0 tx=0).stop 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 msgr2=0x7f5b601213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800c420 tx=0x7f5b5800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 shutdown_connections 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5b4c035450 0x7f5b4c037910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 --2- 172.21.15.96:0/1912157815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b6006f6d0 0x7f5b601213e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 >> 172.21.15.96:0/1912157815 conn(0x7f5b60098a60 msgr2=0x7f5b601392b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 shutdown_connections 2024-09-13T17:42:16.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.017+0000 7f5b661e2640 1 -- 172.21.15.96:0/1912157815 wait complete. 2024-09-13T17:42:16.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.153+0000 7fb6a33e8640 1 Processor -- start 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.153+0000 7fb6a33e8640 1 -- start start 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb69c134670 0x7fb69c134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69c071670 con 0x7fb69c06e5c0 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69c071360 con 0x7fb69c134670 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb69c134670 0x7fb69c134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb69c134670 0x7fb69c134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39554/0 (socket says 172.21.15.96:39554) 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a0983640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb69c06e5c0 0x7fb69c06ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59038/0 (socket says 172.21.15.96:59038) 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 -- 172.21.15.96:0/2492947777 learned_addr learned my addr 172.21.15.96:0/2492947777 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:16.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 -- 172.21.15.96:0/2492947777 >> v1:172.21.15.96:6789/0 conn(0x7fb69c06e5c0 legacy=0x7fb69c06ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 -- 172.21.15.96:0/2492947777 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69c0a7200 con 0x7fb69c134670 2024-09-13T17:42:16.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a1184640 1 --2- 172.21.15.96:0/2492947777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c134670 0x7fb69c134a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb694009e50 tx=0x7fb69401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f419610d8c1deb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb693fff640 1 -- 172.21.15.96:0/2492947777 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb694009200 con 0x7fb69c134670 2024-09-13T17:42:16.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb693fff640 1 -- 172.21.15.96:0/2492947777 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6940a5670 con 0x7fb69c134670 2024-09-13T17:42:16.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb693fff640 1 -- 172.21.15.96:0/2492947777 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6940a5930 con 0x7fb69c134670 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- 172.21.15.96:0/2492947777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c134670 msgr2=0x7fb69c134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/2492947777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c134670 0x7fb69c134a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb694009e50 tx=0x7fb69401c9a0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- 172.21.15.96:0/2492947777 shutdown_connections 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/2492947777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c134670 0x7fb69c134a50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- 172.21.15.96:0/2492947777 >> 172.21.15.96:0/2492947777 conn(0x7fb69c098a60 msgr2=0x7fb69c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- 172.21.15.96:0/2492947777 shutdown_connections 2024-09-13T17:42:16.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.157+0000 7fb6a33e8640 1 -- 172.21.15.96:0/2492947777 wait complete. 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 Processor -- start 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 -- start start 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c1300f0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a1184640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a1184640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39568/0 (socket says 172.21.15.96:39568) 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a1184640 1 -- 172.21.15.96:0/3945842550 learned_addr learned my addr 172.21.15.96:0/3945842550 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a1184640 1 -- 172.21.15.96:0/3945842550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb694009b30 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a1184640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6940195b0 tx=0x7fb694019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6940b8680 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6940b87e0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c099700 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6940b8ac0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c0a4680 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c0a36d0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb6940b8c20 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 0x7fb680037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb6940e31f0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a0983640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 0x7fb680037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.161+0000 7fb6a0983640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 0x7fb680037910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001600 tx=0x7fb68c007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.165+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb6940c07c0 con 0x7fb69c06e5c0 2024-09-13T17:42:16.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.313+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb69c09d520 con 0x7fb69c06e5c0 2024-09-13T17:42:16.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.313+0000 7fb691ffb640 1 -- 172.21.15.96:0/3945842550 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6940bc020 con 0x7fb69c06e5c0 2024-09-13T17:42:16.314 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 msgr2=0x7fb680037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 0x7fb680037910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb68c001600 tx=0x7fb68c007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 msgr2=0x7fb69c1d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb6940195b0 tx=0x7fb694019740 comp rx=0 tx=0).stop 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 shutdown_connections 2024-09-13T17:42:16.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb680035450 0x7fb680037910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 --2- 172.21.15.96:0/3945842550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb69c06e5c0 0x7fb69c1d03c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 >> 172.21.15.96:0/3945842550 conn(0x7fb69c098a60 msgr2=0x7fb69c132f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 shutdown_connections 2024-09-13T17:42:16.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.317+0000 7fb6a33e8640 1 -- 172.21.15.96:0/3945842550 wait complete. 2024-09-13T17:42:16.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdccafef640 1 Processor -- start 2024-09-13T17:42:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdccafef640 1 -- start start 2024-09-13T17:42:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdccafef640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdcc409dc20 0x7fdcc41351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdccafef640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc406e130 con 0x7fdcc406f6d0 2024-09-13T17:42:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdccafef640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc40a6750 con 0x7fdcc409dc20 2024-09-13T17:42:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcbbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdcc409dc20 0x7fdcc41351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcc8d8b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdcc406f6d0 0x7fdcc4134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59050/0 (socket says 172.21.15.96:59050) 2024-09-13T17:42:16.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcc8d8b640 1 -- 172.21.15.96:0/3160050989 learned_addr learned my addr 172.21.15.96:0/3160050989 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:16.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcbbfff640 1 -- 172.21.15.96:0/3160050989 >> v1:172.21.15.96:6789/0 conn(0x7fdcc406f6d0 legacy=0x7fdcc4134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcbbfff640 1 -- 172.21.15.96:0/3160050989 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40712e0 con 0x7fdcc409dc20 2024-09-13T17:42:16.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.453+0000 7fdcbbfff640 1 --2- 172.21.15.96:0/3160050989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc409dc20 0x7fdcc41351c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009740 tx=0x7fdcc001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be815778c1290494 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcbb7fe640 1 -- 172.21.15.96:0/3160050989 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc00ab400 con 0x7fdcc409dc20 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcbb7fe640 1 -- 172.21.15.96:0/3160050989 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc00a7050 con 0x7fdcc409dc20 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcbb7fe640 1 -- 172.21.15.96:0/3160050989 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcc00ab6b0 con 0x7fdcc409dc20 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3160050989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc409dc20 msgr2=0x7fdcc41351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 --2- 172.21.15.96:0/3160050989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc409dc20 0x7fdcc41351c0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009740 tx=0x7fdcc001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3160050989 shutdown_connections 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 --2- 172.21.15.96:0/3160050989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc409dc20 0x7fdcc41351c0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3160050989 >> 172.21.15.96:0/3160050989 conn(0x7fdcc4098a60 msgr2=0x7fdcc409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3160050989 shutdown_connections 2024-09-13T17:42:16.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3160050989 wait complete. 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 Processor -- start 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- start start 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc40a3ce0 con 0x7fdcc406f6d0 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcc8d8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcc8d8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39588/0 (socket says 172.21.15.96:39588) 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcc8d8b640 1 -- 172.21.15.96:0/3594151293 learned_addr learned my addr 172.21.15.96:0/3594151293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:16.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcc8d8b640 1 -- 172.21.15.96:0/3594151293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0009420 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcc8d8b640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c480 tx=0x7fdcb400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcb4005c20 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb4005d80 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc40a1190 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcb40071a0 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.457+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc40a1e30 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fdcb4007300 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc4134ab0 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdcb97fa640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 0x7fdcb0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fdcb4046b70 con 0x7fdcc406f6d0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdcbbfff640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 0x7fdcb0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.461+0000 7fdcbbfff640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 0x7fdcb0037910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00ad000 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.465+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcb40075b0 con 0x7fdcc406f6d0 2024-09-13T17:42:16.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcc406a1b0 con 0x7fdcc406f6d0 2024-09-13T17:42:16.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdcb97fa640 1 -- 172.21.15.96:0/3594151293 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcb4017030 con 0x7fdcc406f6d0 2024-09-13T17:42:16.613 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 msgr2=0x7fdcb0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 0x7fdcb0037910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdcc00ad000 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 msgr2=0x7fdcc41cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400c480 tx=0x7fdcb400c940 comp rx=0 tx=0).stop 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 shutdown_connections 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcb0035450 0x7fdcb0037910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 --2- 172.21.15.96:0/3594151293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcc406f6d0 0x7fdcc41cc970 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 >> 172.21.15.96:0/3594151293 conn(0x7fdcc4098a60 msgr2=0x7fdcc4133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 shutdown_connections 2024-09-13T17:42:16.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.613+0000 7fdccafef640 1 -- 172.21.15.96:0/3594151293 wait complete. 2024-09-13T17:42:16.630 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:16.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0e1669640 1 Processor -- start 2024-09-13T17:42:16.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0e1669640 1 -- start start 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0e1669640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0dc09dc20 0x7fa0dc1351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0e1669640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0dc06e130 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0e1669640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0dc0a6750 con 0x7fa0dc09dc20 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0da7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0dc09dc20 0x7fa0dc1351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0da7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa0dc09dc20 0x7fa0dc1351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39590/0 (socket says 172.21.15.96:39590) 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0daffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa0dc06f6d0 0x7fa0dc134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59064/0 (socket says 172.21.15.96:59064) 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0da7fc640 1 -- 172.21.15.96:0/3140263684 learned_addr learned my addr 172.21.15.96:0/3140263684 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0da7fc640 1 -- 172.21.15.96:0/3140263684 >> v1:172.21.15.96:6789/0 conn(0x7fa0dc06f6d0 legacy=0x7fa0dc134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0da7fc640 1 -- 172.21.15.96:0/3140263684 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0dc0712e0 con 0x7fa0dc09dc20 2024-09-13T17:42:16.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.737+0000 7fa0d9ffb640 1 -- 172.21.15.96:0/3140263684 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974015428 0 0) 0x7fa0dc06e130 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0da7fc640 1 --2- 172.21.15.96:0/3140263684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc09dc20 0x7fa0dc1351c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00097f0 tx=0x7fa0d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8567e966bc7b3f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0d9ffb640 1 -- 172.21.15.96:0/3140263684 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d00ab400 con 0x7fa0dc09dc20 2024-09-13T17:42:16.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0d9ffb640 1 -- 172.21.15.96:0/3140263684 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d00a7050 con 0x7fa0dc09dc20 2024-09-13T17:42:16.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0d9ffb640 1 -- 172.21.15.96:0/3140263684 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d00ab6b0 con 0x7fa0dc09dc20 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- 172.21.15.96:0/3140263684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc09dc20 msgr2=0x7fa0dc1351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3140263684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc09dc20 0x7fa0dc1351c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00097f0 tx=0x7fa0d001c450 comp rx=0 tx=0).stop 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- 172.21.15.96:0/3140263684 shutdown_connections 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3140263684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc09dc20 0x7fa0dc1351c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- 172.21.15.96:0/3140263684 >> 172.21.15.96:0/3140263684 conn(0x7fa0dc098a60 msgr2=0x7fa0dc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- 172.21.15.96:0/3140263684 shutdown_connections 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- 172.21.15.96:0/3140263684 wait complete. 2024-09-13T17:42:16.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 Processor -- start 2024-09-13T17:42:16.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- start start 2024-09-13T17:42:16.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0e1669640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0dc0a3f60 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0daffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39600/0 (socket says 172.21.15.96:39600) 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0daffd640 1 -- 172.21.15.96:0/3037666012 learned_addr learned my addr 172.21.15.96:0/3037666012 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0daffd640 1 -- 172.21.15.96:0/3037666012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d00094d0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.741+0000 7fa0daffd640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00c840 tx=0x7fa0cc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0cc00a660 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc00a7e0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0dc0a3be0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0cc00aac0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0dc0a2c00 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fa0cc00ac20 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0dc134ab0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 0x7fa0c8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:16.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fa0cc046fb0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0da7fc640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 0x7fa0c8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:16.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.745+0000 7fa0da7fc640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 0x7fa0c8037910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00ad000 tx=0x7fa0d0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:16.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.749+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa0cc01a6b0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0dc06a1b0 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0c37fe640 1 -- 172.21.15.96:0/3037666012 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7fa0cc018030 con 0x7fa0dc06f6d0 2024-09-13T17:42:16.941 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:16.941 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:16.941 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:16.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 msgr2=0x7fa0c8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 0x7fa0c8037910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa0d00ad000 tx=0x7fa0d0008040 comp rx=0 tx=0).stop 2024-09-13T17:42:16.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 msgr2=0x7fa0dc1ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00c840 tx=0x7fa0cc00cd00 comp rx=0 tx=0).stop 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 shutdown_connections 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa0c8035450 0x7fa0c8037910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 --2- 172.21.15.96:0/3037666012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0dc06f6d0 0x7fa0dc1ca150 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 >> 172.21.15.96:0/3037666012 conn(0x7fa0dc098a60 msgr2=0x7fa0dc06e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 shutdown_connections 2024-09-13T17:42:16.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:16.941+0000 7fa0e1669640 1 -- 172.21.15.96:0/3037666012 wait complete. 2024-09-13T17:42:16.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:17.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4493fff640 1 Processor -- start 2024-09-13T17:42:17.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4493fff640 1 -- start start 2024-09-13T17:42:17.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4493fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f44941381c0 0x7f449409fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4493fff640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449409df20 con 0x7f44941385a0 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4493fff640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449409efe0 con 0x7f44941381c0 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4492ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f44941381c0 0x7f449409fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f44927fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f44941385a0 0x7f4494134540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59080/0 (socket says 172.21.15.96:59080) 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f44927fc640 1 -- 172.21.15.96:0/4133848529 learned_addr learned my addr 172.21.15.96:0/4133848529 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:17.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4492ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f44941381c0 0x7f449409fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39602/0 (socket says 172.21.15.96:39602) 2024-09-13T17:42:17.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.069+0000 7f4492ffd640 1 -- 172.21.15.96:0/4133848529 >> v1:172.21.15.96:6789/0 conn(0x7f44941385a0 legacy=0x7f4494134540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4492ffd640 1 -- 172.21.15.96:0/4133848529 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449409da50 con 0x7f44941381c0 2024-09-13T17:42:17.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4492ffd640 1 --2- 172.21.15.96:0/4133848529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f449409fa20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4484009740 tx=0x7f448401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885bdd146cb1bcd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4491ffb640 1 -- 172.21.15.96:0/4133848529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44840ab530 con 0x7f44941381c0 2024-09-13T17:42:17.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4491ffb640 1 -- 172.21.15.96:0/4133848529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44840a7050 con 0x7f44941381c0 2024-09-13T17:42:17.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4491ffb640 1 -- 172.21.15.96:0/4133848529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44840ab7e0 con 0x7f44941381c0 2024-09-13T17:42:17.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 -- 172.21.15.96:0/4133848529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 msgr2=0x7f449409fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 --2- 172.21.15.96:0/4133848529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f449409fa20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4484009740 tx=0x7f448401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 -- 172.21.15.96:0/4133848529 shutdown_connections 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 --2- 172.21.15.96:0/4133848529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f449409fa20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 -- 172.21.15.96:0/4133848529 >> 172.21.15.96:0/4133848529 conn(0x7f449406bc30 msgr2=0x7f449406c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 -- 172.21.15.96:0/4133848529 shutdown_connections 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.073+0000 7f4493fff640 1 -- 172.21.15.96:0/4133848529 wait complete. 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 Processor -- start 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 -- start start 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940a1f10 con 0x7f44941381c0 2024-09-13T17:42:17.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4492ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4492ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39616/0 (socket says 172.21.15.96:39616) 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4492ffd640 1 -- 172.21.15.96:0/2896634645 learned_addr learned my addr 172.21.15.96:0/2896634645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4492ffd640 1 -- 172.21.15.96:0/2896634645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009420 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4492ffd640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44840ae040 tx=0x7f4484019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44840b4410 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44840b4570 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44840b4850 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44940a21b0 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44940a30a0 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f44840b2040 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449409fa20 con 0x7f44941381c0 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 0x7f4480037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f44840e1830 con 0x7f44941381c0 2024-09-13T17:42:17.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f44927fc640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 0x7f4480037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.077+0000 7f44927fc640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 0x7f4480037910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4488001610 tx=0x7f4488007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.081+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44840b49b0 con 0x7f44941381c0 2024-09-13T17:42:17.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.237+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f449413dba0 con 0x7f44941381c0 2024-09-13T17:42:17.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.237+0000 7f447bfff640 1 -- 172.21.15.96:0/2896634645 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f44840b0020 con 0x7f44941381c0 2024-09-13T17:42:17.239 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 msgr2=0x7f4480037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 0x7f4480037910 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4488001610 tx=0x7f4488007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 msgr2=0x7f44940a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f44840ae040 tx=0x7f4484019130 comp rx=0 tx=0).stop 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 shutdown_connections 2024-09-13T17:42:17.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4480035450 0x7f4480037910 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 --2- 172.21.15.96:0/2896634645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44941381c0 0x7f44940a2b60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 >> 172.21.15.96:0/2896634645 conn(0x7f449406bc30 msgr2=0x7f4494130db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 shutdown_connections 2024-09-13T17:42:17.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.241+0000 7f4493fff640 1 -- 172.21.15.96:0/2896634645 wait complete. 2024-09-13T17:42:17.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:17.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.361+0000 7f9126687640 1 Processor -- start 2024-09-13T17:42:17.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.361+0000 7f9126687640 1 -- start start 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f912009dc30 0x7f912013ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91201327b0 con 0x7f91201329b0 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912009bd90 con 0x7f912009dc30 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f912009dc30 0x7f912013ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f912009dc30 0x7f912013ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39622/0 (socket says 172.21.15.96:39622) 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 -- 172.21.15.96:0/702992871 learned_addr learned my addr 172.21.15.96:0/702992871 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f91201329b0 0x7f9120132d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59082/0 (socket says 172.21.15.96:59082) 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 -- 172.21.15.96:0/702992871 >> v1:172.21.15.96:6789/0 conn(0x7f91201329b0 legacy=0x7f9120132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 -- 172.21.15.96:0/702992871 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912009b2a0 con 0x7f912009dc30 2024-09-13T17:42:17.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911effd640 1 -- 172.21.15.96:0/702992871 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543748925 0 0) 0x7f91201327b0 con 0x7f91201329b0 2024-09-13T17:42:17.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911f7fe640 1 --2- 172.21.15.96:0/702992871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f912013ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91140097f0 tx=0x7f911401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784d327a4728948 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911effd640 1 -- 172.21.15.96:0/702992871 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91140ad460 con 0x7f912009dc30 2024-09-13T17:42:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911effd640 1 -- 172.21.15.96:0/702992871 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91140ad5c0 con 0x7f912009dc30 2024-09-13T17:42:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f911effd640 1 -- 172.21.15.96:0/702992871 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91140ad880 con 0x7f912009dc30 2024-09-13T17:42:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- 172.21.15.96:0/702992871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 msgr2=0x7f912013ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 --2- 172.21.15.96:0/702992871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f912013ad00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91140097f0 tx=0x7f911401c450 comp rx=0 tx=0).stop 2024-09-13T17:42:17.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- 172.21.15.96:0/702992871 shutdown_connections 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 --2- 172.21.15.96:0/702992871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f912013ad00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- 172.21.15.96:0/702992871 >> 172.21.15.96:0/702992871 conn(0x7f912006ef80 msgr2=0x7f91200713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- 172.21.15.96:0/702992871 shutdown_connections 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.365+0000 7f9126687640 1 -- 172.21.15.96:0/702992871 wait complete. 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 Processor -- start 2024-09-13T17:42:17.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 -- start start 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912009ff40 con 0x7f912009dc30 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39638/0 (socket says 172.21.15.96:39638) 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911ffff640 1 -- 172.21.15.96:0/3180654967 learned_addr learned my addr 172.21.15.96:0/3180654967 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911ffff640 1 -- 172.21.15.96:0/3180654967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91140094d0 con 0x7f912009dc30 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911ffff640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f911800c7d0 tx=0x7f911800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f911800a660 con 0x7f912009dc30 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911800a7e0 con 0x7f912009dc30 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91201396a0 con 0x7f912009dc30 2024-09-13T17:42:17.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f911800aac0 con 0x7f912009dc30 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91200711f0 con 0x7f912009dc30 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f911800ac20 con 0x7f912009dc30 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912006f4f0 con 0x7f912009dc30 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 0x7f910c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9118047620 con 0x7f912009dc30 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911f7fe640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 0x7f910c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.369+0000 7f911f7fe640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 0x7f910c037910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91140af000 tx=0x7f9114008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.373+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9118007070 con 0x7f912009dc30 2024-09-13T17:42:17.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.549+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f912009fc70 con 0x7f912009dc30 2024-09-13T17:42:17.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.549+0000 7f911cff9640 1 -- 172.21.15.96:0/3180654967 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f9118010300 con 0x7f912009dc30 2024-09-13T17:42:17.551 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:17.551 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:17.551 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:17.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 msgr2=0x7f910c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 0x7f910c037910 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91140af000 tx=0x7f9114008040 comp rx=0 tx=0).stop 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 msgr2=0x7f9120139930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f911800c7d0 tx=0x7f911800cc90 comp rx=0 tx=0).stop 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 shutdown_connections 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f910c035450 0x7f910c037910 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 --2- 172.21.15.96:0/3180654967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f912009dc30 0x7f9120139930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 >> 172.21.15.96:0/3180654967 conn(0x7f912006ef80 msgr2=0x7f912012f120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 shutdown_connections 2024-09-13T17:42:17.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.553+0000 7f9126687640 1 -- 172.21.15.96:0/3180654967 wait complete. 2024-09-13T17:42:17.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:42:02.313336+0000', 'modified': '2024-09-13T17:42:04.319061+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4145}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4145': {'gid': 4145, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6829/45340133', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6827', 'nonce': 45340133}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 45340133}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:42:17.768 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:42:17.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:42:17.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f5590145640 1 Processor -- start 2024-09-13T17:42:17.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f5590145640 1 -- start start 2024-09-13T17:42:17.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f5590145640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558809e820 0x7f558813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f5590145640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55880a68b0 con 0x7f5588134e70 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f5590145640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558809ddc0 con 0x7f558809e820 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558d6e0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558809e820 0x7f558813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558d6e0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558809e820 0x7f558813ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39646/0 (socket says 172.21.15.96:39646) 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558d6e0640 1 -- 172.21.15.96:0/1348601543 learned_addr learned my addr 172.21.15.96:0/1348601543 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558dee1640 1 --1- 172.21.15.96:0/1348601543 >> v1:172.21.15.96:6789/0 conn(0x7f5588134e70 0x7f5588135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59086/0 (socket says 172.21.15.96:59086) 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558d6e0640 1 -- 172.21.15.96:0/1348601543 >> v1:172.21.15.96:6789/0 conn(0x7f5588134e70 legacy=0x7f5588135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558d6e0640 1 -- 172.21.15.96:0/1348601543 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5588136970 con 0x7f558809e820 2024-09-13T17:42:17.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.893+0000 7f558cedf640 1 -- 172.21.15.96:0/1348601543 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416905286 0 0) 0x7f55880a68b0 con 0x7f5588134e70 2024-09-13T17:42:17.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558d6e0640 1 --2- 172.21.15.96:0/1348601543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f558813ae50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097f0 tx=0x7f557c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbf1378f636f52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558cedf640 1 -- 172.21.15.96:0/1348601543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557c0ac540 con 0x7f558809e820 2024-09-13T17:42:17.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558cedf640 1 -- 172.21.15.96:0/1348601543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c0ac6a0 con 0x7f558809e820 2024-09-13T17:42:17.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558cedf640 1 -- 172.21.15.96:0/1348601543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557c0ac960 con 0x7f558809e820 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- 172.21.15.96:0/1348601543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 msgr2=0x7f558813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 --2- 172.21.15.96:0/1348601543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f558813ae50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f557c0097f0 tx=0x7f557c01c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- 172.21.15.96:0/1348601543 shutdown_connections 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 --2- 172.21.15.96:0/1348601543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f558813ae50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- 172.21.15.96:0/1348601543 >> 172.21.15.96:0/1348601543 conn(0x7f5588098a60 msgr2=0x7f558809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- 172.21.15.96:0/1348601543 shutdown_connections 2024-09-13T17:42:17.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- 172.21.15.96:0/1348601543 wait complete. 2024-09-13T17:42:17.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 Processor -- start 2024-09-13T17:42:17.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- start start 2024-09-13T17:42:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f5590145640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880a36e0 con 0x7f558809e820 2024-09-13T17:42:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558dee1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558dee1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39654/0 (socket says 172.21.15.96:39654) 2024-09-13T17:42:17.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558dee1640 1 -- 172.21.15.96:0/4241618250 learned_addr learned my addr 172.21.15.96:0/4241618250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.897+0000 7f558dee1640 1 -- 172.21.15.96:0/4241618250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c0094d0 con 0x7f558809e820 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558dee1640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f558000c840 tx=0x7f558000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 -- 172.21.15.96:0/4241618250 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f558000a660 con 0x7f558809e820 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 -- 172.21.15.96:0/4241618250 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000a7e0 con 0x7f558809e820 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558813a4c0 con 0x7f558809e820 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 -- 172.21.15.96:0/4241618250 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f558000aac0 con 0x7f558809e820 2024-09-13T17:42:17.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880a3420 con 0x7f558809e820 2024-09-13T17:42:17.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f559328e000 1 Processor -- start 2024-09-13T17:42:17.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f559328e000 1 -- start start 2024-09-13T17:42:17.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f559328e000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f559328e000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6562fae80 con 0x55b656439710 2024-09-13T17:42:17.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39670/0 (socket says 172.21.15.96:39670) 2024-09-13T17:42:17.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558e6e2640 1 -- 172.21.15.96:0/2812658229 learned_addr learned my addr 172.21.15.96:0/2812658229 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 -- 172.21.15.96:0/4241618250 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f558000ac20 con 0x7f558809e820 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 0x7f5574037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558e6e2640 1 -- 172.21.15.96:0/2812658229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b655f7d890 con 0x55b656439710 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f55867fc640 1 -- 172.21.15.96:0/4241618250 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5580046fd0 con 0x7f558809e820 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558e6e2640 1 --2- 172.21.15.96:0/2812658229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55780096f0 tx=0x7f5578009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24cb76f4c0edf68 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f558dee1640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 0x7f5574037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f556f7fe640 1 -- 172.21.15.96:0/2812658229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557800b8f0 con 0x55b656439710 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f556f7fe640 1 -- 172.21.15.96:0/2812658229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800ba50 con 0x55b656439710 2024-09-13T17:42:17.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.901+0000 7f556f7fe640 1 -- 172.21.15.96:0/2812658229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557800bd30 con 0x55b656439710 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2812658229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 msgr2=0x55b65642f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558dee1640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 0x7f5574037910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f558000afd0 tx=0x7f558000cef0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2812658229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f55780096f0 tx=0x7f5578009bb0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2812658229 shutdown_connections 2024-09-13T17:42:17.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2812658229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65642f8f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2812658229 >> 172.21.15.96:0/2812658229 conn(0x55b65642a230 msgr2=0x55b65642f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2812658229 shutdown_connections 2024-09-13T17:42:17.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2812658229 wait complete. 2024-09-13T17:42:17.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- start start 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 Processor -- start 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b656288af0 con 0x55b656439710 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 10 client.0 _ll_register_callbacks cb 0x55b65641eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39684/0 (socket says 172.21.15.96:39684) 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558e6e2640 1 -- 172.21.15.96:0/2273797129 learned_addr learned my addr 172.21.15.96:0/2273797129 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558e6e2640 1 -- 172.21.15.96:0/2273797129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55780093a0 con 0x55b656439710 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558e6e2640 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f557800b4b0 tx=0x7f55780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df4b7448286c055 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5578011550 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55780116b0 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b65623e5f0 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5578011870 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 10 client.4297 fetch_fsmap learned FSMap version 6 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55b656246ab0 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b65623e5f0 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 6) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f557802b070 con 0x55b656439710 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 10 client.4297 fetch_fsmap finished waiting for FSMap version 6 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 10 client.4297 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 20 client.4297 populate_metadata read hostname 'smithi096' 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65643f130 0x55b656465b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 4 client.4297 mds_command: new command op to 4145 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 --> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b65627e790 con 0x55b65643f130 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558d6e0640 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65643f130 0x55b656465b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f558d6e0640 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65643f130 0x55b656465b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7a46a35f7e3c34b server_cookie=1ac5d5d5799c6d54 in_seq=0 out_seq=0 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 10 client.4297 ms_handle_connect on v2:172.21.15.96:6827/45340133 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 1 -- 172.21.15.96:0/2273797129 <== mds.0 v2:172.21.15.96:6827/45340133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b65627e790 con 0x55b65643f130 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 10 client.4297 handle_command_reply: tid=0 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 client.4297 shutdown 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 msgr2=0x55b65645f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f557800b4b0 tx=0x7f55780046d0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 shutdown_connections 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65643f130 0x55b656465b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 --2- 172.21.15.96:0/2273797129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65645f760 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 >> 172.21.15.96:0/2273797129 conn(0x55b65642a230 msgr2=0x55b656432490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f556cff9640 0 client.4297 ms_handle_reset on v2:172.21.15.96:6827/45340133 2024-09-13T17:42:17.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.905+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 shutdown_connections 2024-09-13T17:42:17.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.909+0000 7f559328e000 1 -- 172.21.15.96:0/2273797129 wait complete. 2024-09-13T17:42:17.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.909+0000 7f559328e000 20 client.4297 trim_cache size 0 max 16384 2024-09-13T17:42:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f559328e000 1 Processor -- start 2024-09-13T17:42:17.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f559328e000 1 -- start start 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f559328e000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f559328e000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6562860c0 con 0x55b656439710 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39694/0 (socket says 172.21.15.96:39694) 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f558e6e2640 1 -- 172.21.15.96:0/3668188350 learned_addr learned my addr 172.21.15.96:0/3668188350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f558e6e2640 1 -- 172.21.15.96:0/3668188350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b65622cc00 con 0x55b656439710 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f558e6e2640 1 --2- 172.21.15.96:0/3668188350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5578009640 tx=0x7f557801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ff10f1436a45b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f556ffff640 1 -- 172.21.15.96:0/3668188350 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557800b4e0 con 0x55b656439710 2024-09-13T17:42:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f556ffff640 1 -- 172.21.15.96:0/3668188350 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557800b8f0 con 0x55b656439710 2024-09-13T17:42:17.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.921+0000 7f556ffff640 1 -- 172.21.15.96:0/3668188350 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5578011cf0 con 0x55b656439710 2024-09-13T17:42:17.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/3668188350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 msgr2=0x55b6564385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/3668188350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5578009640 tx=0x7f557801d040 comp rx=0 tx=0).stop 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/3668188350 shutdown_connections 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/3668188350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b6564385d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/3668188350 >> 172.21.15.96:0/3668188350 conn(0x55b65642a230 msgr2=0x55b656437ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/3668188350 shutdown_connections 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/3668188350 wait complete. 2024-09-13T17:42:17.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- start start 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 Processor -- start 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b6562819b0 con 0x55b656439710 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 10 client.0 _ll_register_callbacks cb 0x55b656230a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558e6e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39708/0 (socket says 172.21.15.96:39708) 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558e6e2640 1 -- 172.21.15.96:0/871737354 learned_addr learned my addr 172.21.15.96:0/871737354 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558e6e2640 1 -- 172.21.15.96:0/871737354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b656246ab0 con 0x55b656439710 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558e6e2640 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5578009bf0 tx=0x7f5578009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab665f62ee810d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:17.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f557803a030 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f557803b070 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b656251460 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55780041b0 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 10 client.4299 fetch_fsmap learned FSMap version 6 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55b655f7dcc0 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b656251460 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 6) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f5578048070 con 0x55b656439710 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 10 client.4299 fetch_fsmap finished waiting for FSMap version 6 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 10 client.4299 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 20 client.4299 populate_metadata read hostname 'smithi096' 2024-09-13T17:42:17.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65649b9e0 0x55b65649bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 4 client.4299 mds_command: new command op to 4145 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 --> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b65624a9c0 con 0x55b65649b9e0 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558d6e0640 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65649b9e0 0x55b65649bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 10 client.4299 ms_handle_connect on v2:172.21.15.96:6827/45340133 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f558d6e0640 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65649b9e0 0x55b65649bdc0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4db6fed91b7a9db server_cookie=cd92d5bf0fd5e991 in_seq=0 out_seq=0 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 1 -- 172.21.15.96:0/871737354 <== mds.0 v2:172.21.15.96:6827/45340133 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55b65624a9c0 con 0x55b65649b9e0 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 10 client.4299 handle_command_reply: tid=0 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 client.4299 shutdown 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 msgr2=0x55b65649b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5578009bf0 tx=0x7f5578009110 comp rx=0 tx=0).stop 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 shutdown_connections 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:6827/45340133,v1:172.21.15.96:6829/45340133] conn(0x55b65649b9e0 0x55b65649bdc0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 --2- 172.21.15.96:0/871737354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b656439710 0x55b65649b4a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f556cff9640 0 client.4299 ms_handle_reset on v2:172.21.15.96:6827/45340133 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 >> 172.21.15.96:0/871737354 conn(0x55b65642a230 msgr2=0x55b6564386e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 shutdown_connections 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 1 -- 172.21.15.96:0/871737354 wait complete. 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f559328e000 20 client.4299 trim_cache size 0 max 16384 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "a54aa7b1-da4c-43a3-b94f-1607e77cbcc4", 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4145, 2024-09-13T17:42:17.932 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 15.614074670000001, 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 6, 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 15, 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 15, 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 21.661279475000001 2024-09-13T17:42:17.933 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 msgr2=0x7f5574037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 0x7f5574037910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f558000afd0 tx=0x7f558000cef0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 msgr2=0x7f55881cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f558000c840 tx=0x7f558000cd00 comp rx=0 tx=0).stop 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 shutdown_connections 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5574035450 0x7f5574037910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 --2- 172.21.15.96:0/4241618250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558809e820 0x7f55881cda00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 >> 172.21.15.96:0/4241618250 conn(0x7f5588098a60 msgr2=0x7f5588134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 shutdown_connections 2024-09-13T17:42:17.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:17.925+0000 7f5590145640 1 -- 172.21.15.96:0/4241618250 wait complete. 2024-09-13T17:42:17.942 INFO:teuthology.run_tasks:Running task sequential... 2024-09-13T17:42:17.951 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-13T17:42:17.951 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-13T17:42:17.952 INFO:teuthology.task.print:Enabling mgr modules 2024-09-13T17:42:17.952 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-13T17:42:17.986 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-13T17:42:17.986 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-13T17:42:17.987 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-13T17:42:17.987 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:42:17.987 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:17.987 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-13T17:42:17.987 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-13T17:42:17.987 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:17.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:17.987 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:42:17.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:17.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-13T17:42:18.049 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-09-13T17:42:18.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:42:18.051 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-13T17:42:18.051 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:42:18.051 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:42:18.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.213+0000 7fd21ff97640 1 Processor -- start 2024-09-13T17:42:18.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.213+0000 7fd21ff97640 1 -- start start 2024-09-13T17:42:18.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21ff97640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd218134e70 0x7fd21809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21ff97640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21809ed60 con 0x7fd218135250 2024-09-13T17:42:18.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21ff97640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21809ddc0 con 0x7fd218134e70 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd218134e70 0x7fd21809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd218134e70 0x7fd21809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39710/0 (socket says 172.21.15.96:39710) 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 -- 172.21.15.96:0/1333268637 learned_addr learned my addr 172.21.15.96:0/1333268637 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21d532640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd218135250 0x7fd218135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59094/0 (socket says 172.21.15.96:59094) 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 -- 172.21.15.96:0/1333268637 >> v1:172.21.15.96:6789/0 conn(0x7fd218135250 legacy=0x7fd218135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 -- 172.21.15.96:0/1333268637 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180a68b0 con 0x7fd218134e70 2024-09-13T17:42:18.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21dd33640 1 --2- 172.21.15.96:0/1333268637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21809e820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097f0 tx=0x7fd21001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466d01f62384c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21cd31640 1 -- 172.21.15.96:0/1333268637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2100ac540 con 0x7fd218134e70 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21cd31640 1 -- 172.21.15.96:0/1333268637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100ac6a0 con 0x7fd218134e70 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21cd31640 1 -- 172.21.15.96:0/1333268637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2100ac960 con 0x7fd218134e70 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21ff97640 1 -- 172.21.15.96:0/1333268637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 msgr2=0x7fd21809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.217+0000 7fd21ff97640 1 --2- 172.21.15.96:0/1333268637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21809e820 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd2100097f0 tx=0x7fd21001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/1333268637 shutdown_connections 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 --2- 172.21.15.96:0/1333268637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21809e820 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/1333268637 >> 172.21.15.96:0/1333268637 conn(0x7fd218098a60 msgr2=0x7fd21809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/1333268637 shutdown_connections 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/1333268637 wait complete. 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 Processor -- start 2024-09-13T17:42:18.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- start start 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21811c930 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21dd33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21dd33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39714/0 (socket says 172.21.15.96:39714) 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21dd33640 1 -- 172.21.15.96:0/499225309 learned_addr learned my addr 172.21.15.96:0/499225309 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21dd33640 1 -- 172.21.15.96:0/499225309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100094d0 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21dd33640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2100190f0 tx=0x7fd210019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2100bb070 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2100b7470 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21811cb60 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2100b7730 con 0x7fd218134e70 2024-09-13T17:42:18.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2180a5120 con 0x7fd218134e70 2024-09-13T17:42:18.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd2100b3070 con 0x7fd218134e70 2024-09-13T17:42:18.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21813b4b0 con 0x7fd218134e70 2024-09-13T17:42:18.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 0x7fd204037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:18.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fd2100e17e0 con 0x7fd218134e70 2024-09-13T17:42:18.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21d532640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 0x7fd204037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:18.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.221+0000 7fd21d532640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 0x7fd204037910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd20c001d00 tx=0x7fd20c007dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:18.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.229+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2100b4380 con 0x7fd218134e70 2024-09-13T17:42:18.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.413+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd21813af00 con 0x7fd218134e70 2024-09-13T17:42:18.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.413+0000 7fd2167fc640 1 -- 172.21.15.96:0/499225309 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd2100b1070 con 0x7fd218134e70 2024-09-13T17:42:18.415 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 msgr2=0x7fd204037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 0x7fd204037910 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd20c001d00 tx=0x7fd20c007dd0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 msgr2=0x7fd21806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd2100190f0 tx=0x7fd210019280 comp rx=0 tx=0).stop 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 shutdown_connections 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd204035450 0x7fd204037910 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 --2- 172.21.15.96:0/499225309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd218134e70 0x7fd21806f6d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:18.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 >> 172.21.15.96:0/499225309 conn(0x7fd218098a60 msgr2=0x7fd218133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:18.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 shutdown_connections 2024-09-13T17:42:18.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:18.417+0000 7fd21ff97640 1 -- 172.21.15.96:0/499225309 wait complete. 2024-09-13T17:42:18.432 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:42:18.432 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:42:18.433 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:42:18.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.434 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:42:18.438 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:42:18.439 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42544sec preferred_lft 42544sec 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:42:18.440 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:18.441 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-13T17:42:18.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-13T17:42:18.441 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:18.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.513 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:42:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:42:18.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.568 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:42:18.633 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:42:18.633 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:42:18.633 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:42:18.633 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:42:18.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:42:18.635 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:18.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.706 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:42:18.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.753 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-13T17:42:18.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.800 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:18.801 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:18.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:42:18.801 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-13T17:42:18.801 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:18.872 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-13T17:42:18.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-13T17:42:18.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-13T17:42:18.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-13T17:42:18.873 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:18.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:18.979 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:18.979 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:18.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-13T17:42:18.979 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-09-13T17:42:18.979 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:18.998 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:42:18.998 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:18.998 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:19.043 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:42:19.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:19.043 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:19.091 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:19.149 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:19.149 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:19.201 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:19.202 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:19.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:19.202 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:19.249 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:42:19.249 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:42:19.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:19.309 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:19.309 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:19.361 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:19.362 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:19.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:19.363 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:19.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-13T17:42:19.369+0000 7ff28a7a2580 -1 init, newargv = 0x55e72ee1a660 newargc=15 2024-09-13T17:42:19.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[33563]: starting ceph client 2024-09-13T17:42:19.374 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[33563]: starting fuse 2024-09-13T17:42:20.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:20.490 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:20.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:20.548 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:20.549 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:20.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.549 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:20.596 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:42:20.596 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:42:20.597 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:42:20.598 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:42:20.716 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.33563.asok 2024-09-13T17:42:20.899 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33563.asok 2024-09-13T17:42:20.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:20.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33563.asok status 2024-09-13T17:42:21.063 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:42:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "33563", 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:42:21.064 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4305, 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4305 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2212810751 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:42:21.065 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2212810751 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4305 192.168.0.1:0/2212810751", 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/2212810751", 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 6, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 15, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:42:21.066 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:42:21.078 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-13T17:42:21.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:42:21.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.197+0000 7f19f138c640 1 Processor -- start 2024-09-13T17:42:21.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.197+0000 7f19f138c640 1 -- start start 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19f138c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19ec134e90 0x7f19ec071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19f138c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec070de0 con 0x7f19ec135270 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19f138c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec071d60 con 0x7f19ec134e90 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19eaffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19ec134e90 0x7f19ec071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19ea7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f19ec135270 0x7f19ec06dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59100/0 (socket says 172.21.15.96:59100) 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19eaffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19ec134e90 0x7f19ec071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39726/0 (socket says 172.21.15.96:39726) 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19ea7fc640 1 -- 172.21.15.96:0/3922561729 learned_addr learned my addr 172.21.15.96:0/3922561729 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19eaffd640 1 -- 172.21.15.96:0/3922561729 >> v1:172.21.15.96:6789/0 conn(0x7f19ec135270 legacy=0x7f19ec06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19e9ffb640 1 -- 172.21.15.96:0/3922561729 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105338534 0 0) 0x7f19ec070de0 con 0x7f19ec135270 2024-09-13T17:42:21.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19eaffd640 1 -- 172.21.15.96:0/3922561729 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec06f110 con 0x7f19ec134e90 2024-09-13T17:42:21.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19eaffd640 1 --2- 172.21.15.96:0/3922561729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec071530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009d90 tx=0x7f19dc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa53886d1af94b10 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:21.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19e9ffb640 1 -- 172.21.15.96:0/3922561729 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0a5650 con 0x7f19ec134e90 2024-09-13T17:42:21.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19e9ffb640 1 -- 172.21.15.96:0/3922561729 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0a57b0 con 0x7f19ec134e90 2024-09-13T17:42:21.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.201+0000 7f19e9ffb640 1 -- 172.21.15.96:0/3922561729 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0a5a70 con 0x7f19ec134e90 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/3922561729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 msgr2=0x7f19ec071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 --2- 172.21.15.96:0/3922561729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec071530 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009d90 tx=0x7f19dc01c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/3922561729 shutdown_connections 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 --2- 172.21.15.96:0/3922561729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec071530 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/3922561729 >> 172.21.15.96:0/3922561729 conn(0x7f19ec098a60 msgr2=0x7f19ec09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/3922561729 shutdown_connections 2024-09-13T17:42:21.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/3922561729 wait complete. 2024-09-13T17:42:21.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 Processor -- start 2024-09-13T17:42:21.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- start start 2024-09-13T17:42:21.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:21.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec133960 con 0x7f19ec134e90 2024-09-13T17:42:21.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19eaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:21.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19eaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39742/0 (socket says 172.21.15.96:39742) 2024-09-13T17:42:21.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19eaffd640 1 -- 172.21.15.96:0/2251701732 learned_addr learned my addr 172.21.15.96:0/2251701732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:21.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19eaffd640 1 -- 172.21.15.96:0/2251701732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009a70 con 0x7f19ec134e90 2024-09-13T17:42:21.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19eaffd640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0194f0 tx=0x7f19dc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0ad570 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0ad6d0 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec0a6010 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0ad9f0 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.205+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec0a55d0 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f19dc0b4050 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec11d2a0 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19e37fe640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 0x7f19cc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f19dc0e22a0 con 0x7f19ec134e90 2024-09-13T17:42:21.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19ea7fc640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 0x7f19cc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:21.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.209+0000 7f19ea7fc640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 0x7f19cc037910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007fd0 tx=0x7f19d4007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:21.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.213+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19dc0ac990 con 0x7f19ec134e90 2024-09-13T17:42:21.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.405+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f19ec06a1b0 con 0x7f19ec134e90 2024-09-13T17:42:21.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.405+0000 7f19e37fe640 1 -- 172.21.15.96:0/2251701732 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f19dc0b2030 con 0x7f19ec134e90 2024-09-13T17:42:21.405 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 msgr2=0x7f19cc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 0x7f19cc037910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f19d4007fd0 tx=0x7f19d4007980 comp rx=0 tx=0).stop 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 msgr2=0x7f19ec1cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0194f0 tx=0x7f19dc019680 comp rx=0 tx=0).stop 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 shutdown_connections 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f19cc035450 0x7f19cc037910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 --2- 172.21.15.96:0/2251701732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ec134e90 0x7f19ec1cc8c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 >> 172.21.15.96:0/2251701732 conn(0x7f19ec098a60 msgr2=0x7f19ec133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 shutdown_connections 2024-09-13T17:42:21.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:21.409+0000 7f19f138c640 1 -- 172.21.15.96:0/2251701732 wait complete. 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:21.423 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:42:21.424 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:42:21.424 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:42:21.424 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:42:21.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.424 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:42:21.429 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:42:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:42:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:42:21.429 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.430 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42541sec preferred_lft 42541sec 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.431 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:42:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:42:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:42:21.432 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.432 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:42:21.484 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:42:21.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.485 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:42:21.535 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:42:21.535 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:42:21.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:42:21.537 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:21.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.609 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:42:21.656 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:42:21.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.657 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-09-13T17:42:21.704 INFO:teuthology.orchestra.run.smithi096.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-13T17:42:21.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 1 2024-09-13T17:42:21.705 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:21.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T17:42:21.831 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 22:14:88:d2:f5:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::2014:88ff:fed2:f548/64 scope link 2024-09-13T17:42:21.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.834 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-13T17:42:21.891 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:42:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:42:21.891 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 22:14:88:d2:f5:48 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::2014:88ff:fed2:f548/64 scope link 2024-09-13T17:42:21.892 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:42:21.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.2/16 2024-09-13T17:42:21.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.1 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-13T17:42:21.894 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:21.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:21.993 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:42:21.993 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:42:21.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:42:21.993 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.1 master ceph-brx 2024-09-13T17:42:21.993 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:42:22.012 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:42:22.013 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:22.013 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:22.063 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:42:22.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:42:22.064 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:22.112 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:22.169 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:22.169 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:22.223 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:22.225 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:22.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.225 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:22.272 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:42:22.272 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:42:22.273 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:42:22.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:22.332 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:22.332 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:22.384 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:22.385 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:22.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:22.386 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:22.397 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:42:22.397+0000 7fc588dd3580 -1 init, newargv = 0x55e06ee439e0 newargc=15 2024-09-13T17:42:22.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[33702]: starting ceph client 2024-09-13T17:42:22.402 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:42:22.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[33702]: starting fuse 2024-09-13T17:42:23.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:42:23.520 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:42:23.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:42:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:42:23.576 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:42:23.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:23.577 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:42:23.624 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:42:23.624 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:42:23.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:42:23.625 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:42:23.626 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:42:23.742 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.33702.asok 2024-09-13T17:42:23.926 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33702.asok 2024-09-13T17:42:23.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:23.926 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33702.asok status 2024-09-13T17:42:24.083 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:42:24.083 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "33702", 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4311, 2024-09-13T17:42:24.084 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4311 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1020454349 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:42:24.085 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1020454349 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4311 192.168.0.2:0/1020454349", 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/1020454349", 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 6, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 15, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:42:24.086 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:42:24.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:24.097 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:42:24.097 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:24.102 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:42:24.103 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:24.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:24.103 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:24.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:24.155 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:42:24.155 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:24.205 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:42:24.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:24.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:24.206 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:24.260 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-13T17:42:24.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adf271640 1 Processor -- start 2024-09-13T17:42:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adf271640 1 -- start start 2024-09-13T17:42:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adf271640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ad809e820 0x7f6ad813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adf271640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad80a68b0 con 0x7f6ad8134e70 2024-09-13T17:42:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adf271640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad809ddc0 con 0x7f6ad809e820 2024-09-13T17:42:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6add00d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6ad8134e70 0x7f6ad8135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43676/0 (socket says 172.21.15.96:43676) 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6add00d640 1 -- 172.21.15.96:0/3281237462 learned_addr learned my addr 172.21.15.96:0/3281237462 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.429+0000 7f6adc80c640 1 --2- 172.21.15.96:0/3281237462 >> v2:172.21.15.96:3300/0 conn(0x7f6ad809e820 0x7f6ad813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6acffff640 1 -- 172.21.15.96:0/3281237462 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533699838 0 0) 0x7f6ad80a68b0 con 0x7f6ad8134e70 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6acffff640 1 -- 172.21.15.96:0/3281237462 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac4003530 con 0x7f6ad8134e70 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adc80c640 1 -- 172.21.15.96:0/3281237462 >> v1:172.21.15.96:6789/0 conn(0x7f6ad8134e70 legacy=0x7f6ad8135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adc80c640 1 -- 172.21.15.96:0/3281237462 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad8136970 con 0x7f6ad809e820 2024-09-13T17:42:24.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adc80c640 1 --2- 172.21.15.96:0/3281237462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad813ae50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009740 tx=0x7f6ac801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c32ade9f5af93c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6acffff640 1 -- 172.21.15.96:0/3281237462 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ac80ab530 con 0x7f6ad809e820 2024-09-13T17:42:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6acffff640 1 -- 172.21.15.96:0/3281237462 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac80a7050 con 0x7f6ad809e820 2024-09-13T17:42:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6acffff640 1 -- 172.21.15.96:0/3281237462 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ac80ab7e0 con 0x7f6ad809e820 2024-09-13T17:42:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 -- 172.21.15.96:0/3281237462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 msgr2=0x7f6ad813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 --2- 172.21.15.96:0/3281237462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad813ae50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009740 tx=0x7f6ac801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 -- 172.21.15.96:0/3281237462 shutdown_connections 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 --2- 172.21.15.96:0/3281237462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad813ae50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 -- 172.21.15.96:0/3281237462 >> 172.21.15.96:0/3281237462 conn(0x7f6ad8098a60 msgr2=0x7f6ad809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 -- 172.21.15.96:0/3281237462 shutdown_connections 2024-09-13T17:42:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.433+0000 7f6adf271640 1 -- 172.21.15.96:0/3281237462 wait complete. 2024-09-13T17:42:24.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 Processor -- start 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 -- start start 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad813a960 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6add00d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6add00d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54536/0 (socket says 172.21.15.96:54536) 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6add00d640 1 -- 172.21.15.96:0/2398459803 learned_addr learned my addr 172.21.15.96:0/2398459803 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6add00d640 1 -- 172.21.15.96:0/2398459803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8009420 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6add00d640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400c450 tx=0x7f6ad400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad4005c20 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ad4005d80 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ad40071a0 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad80a4be0 con 0x7f6ad809e820 2024-09-13T17:42:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad813a410 con 0x7f6ad809e820 2024-09-13T17:42:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad806f350 con 0x7f6ad809e820 2024-09-13T17:42:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f6ad4007300 con 0x7f6ad809e820 2024-09-13T17:42:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 0x7f6abc037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f6ad4046ec0 con 0x7f6ad809e820 2024-09-13T17:42:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adc80c640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 0x7f6abc037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.437+0000 7f6adc80c640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 0x7f6abc037960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80ad000 tx=0x7f6ac8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.441+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6ad40075b0 con 0x7f6ad809e820 2024-09-13T17:42:24.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.601+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ad806f350 con 0x7f6ad809e820 2024-09-13T17:42:24.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.601+0000 7f6acdffb640 1 -- 172.21.15.96:0/2398459803 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f6ad806f350 con 0x7f6ad809e820 2024-09-13T17:42:24.602 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:24.602 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 msgr2=0x7f6abc037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 0x7f6abc037960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80ad000 tx=0x7f6ac8008040 comp rx=0 tx=0).stop 2024-09-13T17:42:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 msgr2=0x7f6ad80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400c450 tx=0x7f6ad400c910 comp rx=0 tx=0).stop 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 shutdown_connections 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6abc0354a0 0x7f6abc037960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 --2- 172.21.15.96:0/2398459803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ad809e820 0x7f6ad80a50f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 >> 172.21.15.96:0/2398459803 conn(0x7f6ad8098a60 msgr2=0x7f6ad8133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 shutdown_connections 2024-09-13T17:42:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.605+0000 7f6adf271640 1 -- 172.21.15.96:0/2398459803 wait complete. 2024-09-13T17:42:24.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:24.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfae204640 1 Processor -- start 2024-09-13T17:42:24.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfae204640 1 -- start start 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfae204640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfa8134e70 0x7fcfa809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfae204640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa809ed60 con 0x7fcfa8135250 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfae204640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa809ddc0 con 0x7fcfa8134e70 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfa8134e70 0x7fcfa809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcfa8135250 0x7fcfa8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43678/0 (socket says 172.21.15.96:43678) 2024-09-13T17:42:24.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa6ffd640 1 -- 172.21.15.96:0/2434392583 learned_addr learned my addr 172.21.15.96:0/2434392583 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa67fc640 1 -- 172.21.15.96:0/2434392583 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879968889 0 0) 0x7fcf9c001120 con 0x7fcfa8135250 2024-09-13T17:42:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa67fc640 1 -- 172.21.15.96:0/2434392583 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003530 con 0x7fcfa8135250 2024-09-13T17:42:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa77fe640 1 -- 172.21.15.96:0/2434392583 >> v1:172.21.15.96:6789/0 conn(0x7fcfa8135250 legacy=0x7fcfa8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.745+0000 7fcfa77fe640 1 -- 172.21.15.96:0/2434392583 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa80a68b0 con 0x7fcfa8134e70 2024-09-13T17:42:24.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa77fe640 1 --2- 172.21.15.96:0/2434392583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa809e820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009740 tx=0x7fcf9801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcef4df7d215e4cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa67fc640 1 -- 172.21.15.96:0/2434392583 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf980ab530 con 0x7fcfa8134e70 2024-09-13T17:42:24.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa67fc640 1 -- 172.21.15.96:0/2434392583 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf980a7050 con 0x7fcfa8134e70 2024-09-13T17:42:24.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa67fc640 1 -- 172.21.15.96:0/2434392583 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf980ab7c0 con 0x7fcfa8134e70 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- 172.21.15.96:0/2434392583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 msgr2=0x7fcfa809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 --2- 172.21.15.96:0/2434392583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa809e820 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009740 tx=0x7fcf9801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- 172.21.15.96:0/2434392583 shutdown_connections 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 --2- 172.21.15.96:0/2434392583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa809e820 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- 172.21.15.96:0/2434392583 >> 172.21.15.96:0/2434392583 conn(0x7fcfa8098a60 msgr2=0x7fcfa809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- 172.21.15.96:0/2434392583 shutdown_connections 2024-09-13T17:42:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- 172.21.15.96:0/2434392583 wait complete. 2024-09-13T17:42:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 Processor -- start 2024-09-13T17:42:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- start start 2024-09-13T17:42:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfae204640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa80a4ed0 con 0x7fcfa8134e70 2024-09-13T17:42:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54546/0 (socket says 172.21.15.96:54546) 2024-09-13T17:42:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.749+0000 7fcfa77fe640 1 -- 172.21.15.96:0/963016841 learned_addr learned my addr 172.21.15.96:0/963016841 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfa77fe640 1 -- 172.21.15.96:0/963016841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98009420 con 0x7fcfa8134e70 2024-09-13T17:42:24.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfa77fe640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcf980ae040 tx=0x7fcf980191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf98003960 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf98003ac0 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa809a730 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf98003d80 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa811e230 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcf980b2040 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa80a5870 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 0x7fcf94037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fcf980e16c0 con 0x7fcfa8134e70 2024-09-13T17:42:24.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfa6ffd640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 0x7fcf94037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:24.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.753+0000 7fcfa6ffd640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 0x7fcf94037910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c001280 tx=0x7fcf9c001130 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:24.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.757+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf980c5530 con 0x7fcfa8134e70 2024-09-13T17:42:24.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcfa811e020 con 0x7fcfa8134e70 2024-09-13T17:42:24.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcf8ffff640 1 -- 172.21.15.96:0/963016841 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf980b8070 con 0x7fcfa8134e70 2024-09-13T17:42:24.912 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:24.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 msgr2=0x7fcf94037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 0x7fcf94037910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c001280 tx=0x7fcf9c001130 comp rx=0 tx=0).stop 2024-09-13T17:42:24.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 msgr2=0x7fcfa80a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcf980ae040 tx=0x7fcf980191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 shutdown_connections 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcf94035450 0x7fcf94037910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 --2- 172.21.15.96:0/963016841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfa8134e70 0x7fcfa80a3970 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.913+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 >> 172.21.15.96:0/963016841 conn(0x7fcfa8098a60 msgr2=0x7fcfa8134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.917+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 shutdown_connections 2024-09-13T17:42:24.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:24.917+0000 7fcfae204640 1 -- 172.21.15.96:0/963016841 wait complete. 2024-09-13T17:42:24.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.045+0000 7ff7853eb640 1 Processor -- start 2024-09-13T17:42:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.045+0000 7ff7853eb640 1 -- start start 2024-09-13T17:42:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff78006f6d0 0x7ff780134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78006e130 con 0x7ff78009dc20 2024-09-13T17:42:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7800a6750 con 0x7ff78006f6d0 2024-09-13T17:42:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77e7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff78009dc20 0x7ff780134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43684/0 (socket says 172.21.15.96:43684) 2024-09-13T17:42:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77e7fc640 1 -- 172.21.15.96:0/2026953295 learned_addr learned my addr 172.21.15.96:0/2026953295 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77effd640 1 --2- 172.21.15.96:0/2026953295 >> v2:172.21.15.96:3300/0 conn(0x7ff78006f6d0 0x7ff780134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77dffb640 1 -- 172.21.15.96:0/2026953295 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738523234 0 0) 0x7ff78006e130 con 0x7ff78009dc20 2024-09-13T17:42:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77dffb640 1 -- 172.21.15.96:0/2026953295 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff768003530 con 0x7ff78009dc20 2024-09-13T17:42:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77effd640 1 -- 172.21.15.96:0/2026953295 >> v1:172.21.15.96:6789/0 conn(0x7ff78009dc20 legacy=0x7ff780134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77effd640 1 -- 172.21.15.96:0/2026953295 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800712e0 con 0x7ff78006f6d0 2024-09-13T17:42:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77effd640 1 --2- 172.21.15.96:0/2026953295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff780134ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff774009740 tx=0x7ff77401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be21004bd8d9be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77dffb640 1 -- 172.21.15.96:0/2026953295 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7740ab400 con 0x7ff78006f6d0 2024-09-13T17:42:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77dffb640 1 -- 172.21.15.96:0/2026953295 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7740a7050 con 0x7ff78006f6d0 2024-09-13T17:42:25.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff77dffb640 1 -- 172.21.15.96:0/2026953295 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7740ab6b0 con 0x7ff78006f6d0 2024-09-13T17:42:25.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- 172.21.15.96:0/2026953295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 msgr2=0x7ff780134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 --2- 172.21.15.96:0/2026953295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff780134ab0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff774009740 tx=0x7ff77401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- 172.21.15.96:0/2026953295 shutdown_connections 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 --2- 172.21.15.96:0/2026953295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff780134ab0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- 172.21.15.96:0/2026953295 >> 172.21.15.96:0/2026953295 conn(0x7ff780098a60 msgr2=0x7ff78009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- 172.21.15.96:0/2026953295 shutdown_connections 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.049+0000 7ff7853eb640 1 -- 172.21.15.96:0/2026953295 wait complete. 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 Processor -- start 2024-09-13T17:42:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 -- start start 2024-09-13T17:42:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78009f5a0 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54566/0 (socket says 172.21.15.96:54566) 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77effd640 1 -- 172.21.15.96:0/3178714027 learned_addr learned my addr 172.21.15.96:0/3178714027 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77effd640 1 -- 172.21.15.96:0/3178714027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff774009420 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77effd640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff774019fc0 tx=0x7ff77401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7740abe10 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7740b7450 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7740b76f0 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7800a2280 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7800a1370 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff7740b7850 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78006eb00 con 0x7ff78006f6d0 2024-09-13T17:42:25.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 0x7ff76c0378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff7740e1f60 con 0x7ff78006f6d0 2024-09-13T17:42:25.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77e7fc640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 0x7ff76c0378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.053+0000 7ff77e7fc640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 0x7ff76c0378c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.057+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff774113db0 con 0x7ff78006f6d0 2024-09-13T17:42:25.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.209+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff78006a1b0 con 0x7ff78006f6d0 2024-09-13T17:42:25.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.209+0000 7ff7677fe640 1 -- 172.21.15.96:0/3178714027 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7740bb020 con 0x7ff78006f6d0 2024-09-13T17:42:25.210 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 msgr2=0x7ff76c0378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 0x7ff76c0378c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff770007fd0 tx=0x7ff770007980 comp rx=0 tx=0).stop 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 msgr2=0x7ff7800a1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff774019fc0 tx=0x7ff77401abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 shutdown_connections 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff76c035400 0x7ff76c0378c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 --2- 172.21.15.96:0/3178714027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff78006f6d0 0x7ff7800a1a50 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 >> 172.21.15.96:0/3178714027 conn(0x7ff780098a60 msgr2=0x7ff78009ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 shutdown_connections 2024-09-13T17:42:25.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.213+0000 7ff7853eb640 1 -- 172.21.15.96:0/3178714027 wait complete. 2024-09-13T17:42:25.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:25.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.337+0000 7fe73b828640 1 Processor -- start 2024-09-13T17:42:25.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.337+0000 7fe73b828640 1 -- start start 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe73b828640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe734137490 0x7fe734139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe73b828640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe734071670 con 0x7fe734134670 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe73b828640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe734071360 con 0x7fe734137490 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe738dc3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe734137490 0x7fe734139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe7395c4640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe734134670 0x7fe734134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43686/0 (socket says 172.21.15.96:43686) 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe7395c4640 1 -- 172.21.15.96:0/795135154 learned_addr learned my addr 172.21.15.96:0/795135154 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:25.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe738dc3640 1 -- 172.21.15.96:0/795135154 >> v1:172.21.15.96:6789/0 conn(0x7fe734134670 legacy=0x7fe734134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe738dc3640 1 -- 172.21.15.96:0/795135154 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7340a7200 con 0x7fe734137490 2024-09-13T17:42:25.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe738dc3640 1 --2- 172.21.15.96:0/795135154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734137490 0x7fe734139880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe730009740 tx=0x7fe73001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85a7dafa8795d08d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe72bfff640 1 -- 172.21.15.96:0/795135154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7300ab530 con 0x7fe734137490 2024-09-13T17:42:25.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe72bfff640 1 -- 172.21.15.96:0/795135154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7300a7050 con 0x7fe734137490 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe72bfff640 1 -- 172.21.15.96:0/795135154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7300ab7e0 con 0x7fe734137490 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.341+0000 7fe73b828640 1 -- 172.21.15.96:0/795135154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734137490 msgr2=0x7fe734139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 --2- 172.21.15.96:0/795135154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734137490 0x7fe734139880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe730009740 tx=0x7fe73001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/795135154 shutdown_connections 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 --2- 172.21.15.96:0/795135154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734137490 0x7fe734139880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/795135154 >> 172.21.15.96:0/795135154 conn(0x7fe734098a60 msgr2=0x7fe73409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/795135154 shutdown_connections 2024-09-13T17:42:25.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/795135154 wait complete. 2024-09-13T17:42:25.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 Processor -- start 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- start start 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7340a41d0 con 0x7fe734134670 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe7395c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe7395c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54584/0 (socket says 172.21.15.96:54584) 2024-09-13T17:42:25.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe7395c4640 1 -- 172.21.15.96:0/1536646366 learned_addr learned my addr 172.21.15.96:0/1536646366 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:25.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe7395c4640 1 -- 172.21.15.96:0/1536646366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009420 con 0x7fe734134670 2024-09-13T17:42:25.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe7395c4640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c480 tx=0x7fe72400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe724005c20 con 0x7fe734134670 2024-09-13T17:42:25.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724005d80 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7240071a0 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7340a1fd0 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7340a3250 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe724007300 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe734133ab0 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 0x7fe718037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe724046bc0 con 0x7fe734134670 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe738dc3640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 0x7fe718037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.345+0000 7fe738dc3640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 0x7fe718037910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7300ad000 tx=0x7fe7300a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.353+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe7240075b0 con 0x7fe734134670 2024-09-13T17:42:25.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.513+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe734134a50 con 0x7fe734134670 2024-09-13T17:42:25.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.513+0000 7fe729ffb640 1 -- 172.21.15.96:0/1536646366 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe724017070 con 0x7fe734134670 2024-09-13T17:42:25.514 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:25.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 msgr2=0x7fe718037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 0x7fe718037910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7300ad000 tx=0x7fe7300a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 msgr2=0x7fe7340a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe72400c480 tx=0x7fe72400c940 comp rx=0 tx=0).stop 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 shutdown_connections 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe718035450 0x7fe718037910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 --2- 172.21.15.96:0/1536646366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe734134670 0x7fe7340a3c90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 >> 172.21.15.96:0/1536646366 conn(0x7fe734098a60 msgr2=0x7fe734130e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 shutdown_connections 2024-09-13T17:42:25.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.517+0000 7fe73b828640 1 -- 172.21.15.96:0/1536646366 wait complete. 2024-09-13T17:42:25.531 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.637+0000 7f4d15e5c640 1 Processor -- start 2024-09-13T17:42:25.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d15e5c640 1 -- start start 2024-09-13T17:42:25.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d15e5c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4d1009d690 0x7f4d10138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d15e5c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d10138050 con 0x7f4d101387d0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d15e5c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1009c410 con 0x7f4d1009d690 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4d1009d690 0x7f4d10138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4d101387d0 0x7f4d10134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43694/0 (socket says 172.21.15.96:43694) 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0effd640 1 -- 172.21.15.96:0/2064553937 learned_addr learned my addr 172.21.15.96:0/2064553937 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0f7fe640 1 -- 172.21.15.96:0/2064553937 >> v1:172.21.15.96:6789/0 conn(0x7f4d101387d0 legacy=0x7f4d10134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0f7fe640 1 -- 172.21.15.96:0/2064553937 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1009cba0 con 0x7f4d1009d690 2024-09-13T17:42:25.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.641+0000 7f4d0e7fc640 1 -- 172.21.15.96:0/2064553937 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4057575028 0 0) 0x7f4d000014a0 con 0x7f4d101387d0 2024-09-13T17:42:25.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 --2- 172.21.15.96:0/2064553937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d10138290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009740 tx=0x7f4d0801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cba8bbf83743fe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0e7fc640 1 -- 172.21.15.96:0/2064553937 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d080a8840 con 0x7f4d1009d690 2024-09-13T17:42:25.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0e7fc640 1 -- 172.21.15.96:0/2064553937 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d080a89a0 con 0x7f4d1009d690 2024-09-13T17:42:25.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0e7fc640 1 -- 172.21.15.96:0/2064553937 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d080a8c60 con 0x7f4d1009d690 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- 172.21.15.96:0/2064553937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 msgr2=0x7f4d10138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/2064553937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d10138290 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d08009740 tx=0x7f4d0801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- 172.21.15.96:0/2064553937 shutdown_connections 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/2064553937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d10138290 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- 172.21.15.96:0/2064553937 >> 172.21.15.96:0/2064553937 conn(0x7f4d1006bc60 msgr2=0x7f4d1006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- 172.21.15.96:0/2064553937 shutdown_connections 2024-09-13T17:42:25.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- 172.21.15.96:0/2064553937 wait complete. 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 Processor -- start 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- start start 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d15e5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d100a61a0 con 0x7f4d1009d690 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54606/0 (socket says 172.21.15.96:54606) 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 -- 172.21.15.96:0/381038385 learned_addr learned my addr 172.21.15.96:0/381038385 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 -- 172.21.15.96:0/381038385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d08009420 con 0x7f4d1009d690 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.645+0000 7f4d0f7fe640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d0801b820 tx=0x7f4d0801d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d08004150 con 0x7f4d1009d690 2024-09-13T17:42:25.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d080042b0 con 0x7f4d1009d690 2024-09-13T17:42:25.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d100a5ee0 con 0x7f4d1009d690 2024-09-13T17:42:25.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d080bc720 con 0x7f4d1009d690 2024-09-13T17:42:25.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1009f420 con 0x7f4d1009d690 2024-09-13T17:42:25.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1009bc00 con 0x7f4d1009d690 2024-09-13T17:42:25.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f4d080bc880 con 0x7f4d1009d690 2024-09-13T17:42:25.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 0x7f4cfc0378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f4d080e5f60 con 0x7f4d1009d690 2024-09-13T17:42:25.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4d0effd640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 0x7f4cfc0378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.649+0000 7f4d0effd640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 0x7f4cfc0378c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000f90 tx=0x7f4d000015e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.653+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4d08111db0 con 0x7f4d1009d690 2024-09-13T17:42:25.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.797+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d100a62e0 con 0x7f4d1009d690 2024-09-13T17:42:25.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4cf7fff640 1 -- 172.21.15.96:0/381038385 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f4d100a62e0 con 0x7f4d1009d690 2024-09-13T17:42:25.800 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:25.800 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 msgr2=0x7f4cfc0378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 0x7f4cfc0378c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4d00000f90 tx=0x7f4d000015e0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 msgr2=0x7f4d100a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d0801b820 tx=0x7f4d0801d450 comp rx=0 tx=0).stop 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 shutdown_connections 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4cfc035400 0x7f4cfc0378c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 --2- 172.21.15.96:0/381038385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d1009d690 0x7f4d100a6920 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 >> 172.21.15.96:0/381038385 conn(0x7f4d1006bc60 msgr2=0x7f4d1009e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 shutdown_connections 2024-09-13T17:42:25.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.801+0000 7f4d15e5c640 1 -- 172.21.15.96:0/381038385 wait complete. 2024-09-13T17:42:25.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:25.817 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb5ba2d640 1 Processor -- start 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb5ba2d640 1 -- start start 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb5ba2d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb54133600 0x7fcb541339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb5ba2d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5406dad0 con 0x7fcb54133f20 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb5ba2d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5406eb80 con 0x7fcb54133600 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb597c9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb54133600 0x7fcb541339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb597c9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb54133600 0x7fcb541339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54618/0 (socket says 172.21.15.96:54618) 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb597c9640 1 -- 172.21.15.96:0/531091244 learned_addr learned my addr 172.21.15.96:0/531091244 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:25.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb58fc8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcb54133f20 0x7fcb54134340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43706/0 (socket says 172.21.15.96:43706) 2024-09-13T17:42:25.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb597c9640 1 -- 172.21.15.96:0/531091244 >> v1:172.21.15.96:6789/0 conn(0x7fcb54133f20 legacy=0x7fcb54134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.929+0000 7fcb597c9640 1 -- 172.21.15.96:0/531091244 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540a6490 con 0x7fcb54133600 2024-09-13T17:42:25.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 --2- 172.21.15.96:0/531091244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541339e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0097f0 tx=0x7fcb4c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd00f55c9fe2534 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb4bfff640 1 -- 172.21.15.96:0/531091244 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c0ab470 con 0x7fcb54133600 2024-09-13T17:42:25.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb4bfff640 1 -- 172.21.15.96:0/531091244 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0ab5d0 con 0x7fcb54133600 2024-09-13T17:42:25.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb4bfff640 1 -- 172.21.15.96:0/531091244 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c0ab890 con 0x7fcb54133600 2024-09-13T17:42:25.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/531091244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 msgr2=0x7fcb541339e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:25.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/531091244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541339e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c0097f0 tx=0x7fcb4c01c450 comp rx=0 tx=0).stop 2024-09-13T17:42:25.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/531091244 shutdown_connections 2024-09-13T17:42:25.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/531091244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541339e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:25.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/531091244 >> 172.21.15.96:0/531091244 conn(0x7fcb5406a150 msgr2=0x7fcb5406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:25.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/531091244 shutdown_connections 2024-09-13T17:42:25.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/531091244 wait complete. 2024-09-13T17:42:25.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 Processor -- start 2024-09-13T17:42:25.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- start start 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5409ff70 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54626/0 (socket says 172.21.15.96:54626) 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 -- 172.21.15.96:0/3778614307 learned_addr learned my addr 172.21.15.96:0/3778614307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 -- 172.21.15.96:0/3778614307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c0094d0 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb597c9640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c019060 tx=0x7fcb4c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c0ab470 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0b4600 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5409f050 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb4c0b48a0 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5409ed40 con 0x7fcb54133600 2024-09-13T17:42:25.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcb4c0b4a00 con 0x7fcb54133600 2024-09-13T17:42:25.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb54073570 con 0x7fcb54133600 2024-09-13T17:42:25.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 0x7fcb38037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:25.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fcb4c0e2280 con 0x7fcb54133600 2024-09-13T17:42:25.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.933+0000 7fcb58fc8640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 0x7fcb38037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:25.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.937+0000 7fcb58fc8640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 0x7fcb38037910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001600 tx=0x7fcb44007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:25.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:25.941+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb4c0aa9f0 con 0x7fcb54133600 2024-09-13T17:42:26.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb5406fae0 con 0x7fcb54133600 2024-09-13T17:42:26.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb49ffb640 1 -- 172.21.15.96:0/3778614307 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb5406fae0 con 0x7fcb54133600 2024-09-13T17:42:26.092 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 msgr2=0x7fcb38037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 0x7fcb38037910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcb44001600 tx=0x7fcb44007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 msgr2=0x7fcb541cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c019060 tx=0x7fcb4c01ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 shutdown_connections 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcb38035450 0x7fcb38037910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 --2- 172.21.15.96:0/3778614307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb54133600 0x7fcb541cd520 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 >> 172.21.15.96:0/3778614307 conn(0x7fcb5406a150 msgr2=0x7fcb54070040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 shutdown_connections 2024-09-13T17:42:26.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.093+0000 7fcb5ba2d640 1 -- 172.21.15.96:0/3778614307 wait complete. 2024-09-13T17:42:26.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:26.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:42:26.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.221+0000 7f78c1eb4640 1 Processor -- start 2024-09-13T17:42:26.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.221+0000 7f78c1eb4640 1 -- start start 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f78bc09dc20 0x7f78bc1351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78bc06e130 con 0x7f78bc06f6d0 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78bc0a6750 con 0x7f78bc09dc20 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78bb7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f78bc06f6d0 0x7f78bc134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43714/0 (socket says 172.21.15.96:43714) 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78bb7fe640 1 -- 172.21.15.96:0/2196422171 learned_addr learned my addr 172.21.15.96:0/2196422171 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78b3fff640 1 --2- 172.21.15.96:0/2196422171 >> v2:172.21.15.96:3300/0 conn(0x7f78bc09dc20 0x7f78bc1351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78baffd640 1 -- 172.21.15.96:0/2196422171 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1289668604 0 0) 0x7f78bc06e130 con 0x7f78bc06f6d0 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78baffd640 1 -- 172.21.15.96:0/2196422171 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78a8005740 con 0x7f78bc06f6d0 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78b3fff640 1 -- 172.21.15.96:0/2196422171 >> v1:172.21.15.96:6789/0 conn(0x7f78bc06f6d0 legacy=0x7f78bc134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78b3fff640 1 -- 172.21.15.96:0/2196422171 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0712e0 con 0x7f78bc09dc20 2024-09-13T17:42:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78b3fff640 1 --2- 172.21.15.96:0/2196422171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc09dc20 0x7f78bc1351c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009740 tx=0x7f78a401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d33eada6253e8b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78baffd640 1 -- 172.21.15.96:0/2196422171 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a40ab400 con 0x7f78bc09dc20 2024-09-13T17:42:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78baffd640 1 -- 172.21.15.96:0/2196422171 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a40a7050 con 0x7f78bc09dc20 2024-09-13T17:42:26.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78baffd640 1 -- 172.21.15.96:0/2196422171 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78a40ab6b0 con 0x7f78bc09dc20 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- 172.21.15.96:0/2196422171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc09dc20 msgr2=0x7f78bc1351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/2196422171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc09dc20 0x7f78bc1351c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009740 tx=0x7f78a401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- 172.21.15.96:0/2196422171 shutdown_connections 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/2196422171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc09dc20 0x7f78bc1351c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- 172.21.15.96:0/2196422171 >> 172.21.15.96:0/2196422171 conn(0x7f78bc098a60 msgr2=0x7f78bc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- 172.21.15.96:0/2196422171 shutdown_connections 2024-09-13T17:42:26.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.225+0000 7f78c1eb4640 1 -- 172.21.15.96:0/2196422171 wait complete. 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 Processor -- start 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 -- start start 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78bc09ab80 con 0x7f78bc06f6d0 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78bb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54652/0 (socket says 172.21.15.96:54652) 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78bb7fe640 1 -- 172.21.15.96:0/3470966774 learned_addr learned my addr 172.21.15.96:0/3470966774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78bb7fe640 1 -- 172.21.15.96:0/3470966774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009420 con 0x7f78bc06f6d0 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78bb7fe640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00c450 tx=0x7f78ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78ac005c20 con 0x7f78bc06f6d0 2024-09-13T17:42:26.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac005d80 con 0x7f78bc06f6d0 2024-09-13T17:42:26.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78bc11cd50 con 0x7f78bc06f6d0 2024-09-13T17:42:26.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78ac0071a0 con 0x7f78bc06f6d0 2024-09-13T17:42:26.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78bc11dda0 con 0x7f78bc06f6d0 2024-09-13T17:42:26.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f78ac007300 con 0x7f78bc06f6d0 2024-09-13T17:42:26.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc06f2e0 con 0x7f78bc06f6d0 2024-09-13T17:42:26.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 0x7f789c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f78ac046a00 con 0x7f78bc06f6d0 2024-09-13T17:42:26.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b3fff640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 0x7f789c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.229+0000 7f78b3fff640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 0x7f789c037910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ad000 tx=0x7f78a4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.233+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f78ac0108f0 con 0x7f78bc06f6d0 2024-09-13T17:42:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.393+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78bc11d6d0 con 0x7f78bc06f6d0 2024-09-13T17:42:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.393+0000 7f78b8ff9640 1 -- 172.21.15.96:0/3470966774 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78ac017030 con 0x7f78bc06f6d0 2024-09-13T17:42:26.395 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 msgr2=0x7f789c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 0x7f789c037910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78a40ad000 tx=0x7f78a4008040 comp rx=0 tx=0).stop 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 msgr2=0x7f78bc11e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00c450 tx=0x7f78ac00c910 comp rx=0 tx=0).stop 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 shutdown_connections 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f789c035450 0x7f789c037910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 --2- 172.21.15.96:0/3470966774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78bc06f6d0 0x7f78bc11e490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 >> 172.21.15.96:0/3470966774 conn(0x7f78bc098a60 msgr2=0x7f78bc098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 shutdown_connections 2024-09-13T17:42:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.397+0000 7f78c1eb4640 1 -- 172.21.15.96:0/3470966774 wait complete. 2024-09-13T17:42:26.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:26.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a7e7d640 1 Processor -- start 2024-09-13T17:42:26.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a7e7d640 1 -- start start 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a7e7d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f58a0138590 0x7f58a013d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a7e7d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58a006ca30 con 0x7f58a01381b0 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a7e7d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58a00a02e0 con 0x7f58a0138590 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a5c19640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f58a01381b0 0x7f58a00a0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43716/0 (socket says 172.21.15.96:43716) 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a5c19640 1 -- 172.21.15.96:0/2707463804 learned_addr learned my addr 172.21.15.96:0/2707463804 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.577+0000 7f58a5418640 1 --2- 172.21.15.96:0/2707463804 >> v2:172.21.15.96:3300/0 conn(0x7f58a0138590 0x7f58a013d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a5418640 1 -- 172.21.15.96:0/2707463804 >> v1:172.21.15.96:6789/0 conn(0x7f58a01381b0 legacy=0x7f58a00a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a5418640 1 -- 172.21.15.96:0/2707463804 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a00a05a0 con 0x7f58a0138590 2024-09-13T17:42:26.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a5418640 1 --2- 172.21.15.96:0/2707463804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a0138590 0x7f58a013d010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5890009740 tx=0x7f589001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d342e36a4e257d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a4c17640 1 -- 172.21.15.96:0/2707463804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58900ab530 con 0x7f58a0138590 2024-09-13T17:42:26.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a4c17640 1 -- 172.21.15.96:0/2707463804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58900a7050 con 0x7f58a0138590 2024-09-13T17:42:26.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a4c17640 1 -- 172.21.15.96:0/2707463804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58900ab7e0 con 0x7f58a0138590 2024-09-13T17:42:26.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2707463804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a0138590 msgr2=0x7f58a013d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2707463804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a0138590 0x7f58a013d010 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5890009740 tx=0x7f589001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:26.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2707463804 shutdown_connections 2024-09-13T17:42:26.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2707463804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a0138590 0x7f58a013d010 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2707463804 >> 172.21.15.96:0/2707463804 conn(0x7f58a006ef80 msgr2=0x7f58a00713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2707463804 shutdown_connections 2024-09-13T17:42:26.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.581+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2707463804 wait complete. 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 Processor -- start 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 -- start start 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a00a46e0 con 0x7f58a01381b0 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5c19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5c19640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54680/0 (socket says 172.21.15.96:54680) 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5c19640 1 -- 172.21.15.96:0/2703712478 learned_addr learned my addr 172.21.15.96:0/2703712478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5c19640 1 -- 172.21.15.96:0/2703712478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5890009420 con 0x7f58a01381b0 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5c19640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c380 tx=0x7f589c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f589c005c20 con 0x7f58a01381b0 2024-09-13T17:42:26.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f589c005d80 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a006f580 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f589c007200 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a00a26c0 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a00a4100 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f589c007360 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 0x7f5884037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f589c047480 con 0x7f58a01381b0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.585+0000 7f58a5418640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 0x7f5884037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.589+0000 7f58a5418640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 0x7f5884037910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58900ad000 tx=0x7f58900a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.589+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f589c012400 con 0x7f58a01381b0 2024-09-13T17:42:26.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.749+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58a00711a0 con 0x7f58a01381b0 2024-09-13T17:42:26.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.749+0000 7f58967fc640 1 -- 172.21.15.96:0/2703712478 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f589c0232f0 con 0x7f58a01381b0 2024-09-13T17:42:26.750 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 msgr2=0x7f5884037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 0x7f5884037910 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58900ad000 tx=0x7f58900a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 msgr2=0x7f58a00a49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f589c00c380 tx=0x7f589c00c840 comp rx=0 tx=0).stop 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 shutdown_connections 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5884035450 0x7f5884037910 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 --2- 172.21.15.96:0/2703712478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58a01381b0 0x7f58a00a49e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 >> 172.21.15.96:0/2703712478 conn(0x7f58a006ef80 msgr2=0x7f58a0136e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 shutdown_connections 2024-09-13T17:42:26.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.753+0000 7f58a7e7d640 1 -- 172.21.15.96:0/2703712478 wait complete. 2024-09-13T17:42:26.768 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c344b0640 1 Processor -- start 2024-09-13T17:42:26.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c344b0640 1 -- start start 2024-09-13T17:42:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c344b0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c2c134e70 0x7f8c2c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c344b0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2c09ed60 con 0x7f8c2c135250 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c344b0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c2c09ddc0 con 0x7f8c2c134e70 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c3224c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c2c134e70 0x7f8c2c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c31a4b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8c2c135250 0x7f8c2c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43720/0 (socket says 172.21.15.96:43720) 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.881+0000 7f8c31a4b640 1 -- 172.21.15.96:0/3299099714 learned_addr learned my addr 172.21.15.96:0/3299099714 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.885+0000 7f8c3224c640 1 -- 172.21.15.96:0/3299099714 >> v1:172.21.15.96:6789/0 conn(0x7f8c2c135250 legacy=0x7f8c2c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.885+0000 7f8c3124a640 1 -- 172.21.15.96:0/3299099714 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649472931 0 0) 0x7f8c20001120 con 0x7f8c2c135250 2024-09-13T17:42:26.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.885+0000 7f8c3224c640 1 -- 172.21.15.96:0/3299099714 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c0a68b0 con 0x7f8c2c134e70 2024-09-13T17:42:26.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c3224c640 1 --2- 172.21.15.96:0/3299099714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c09e820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009740 tx=0x7f8c2401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efa2560756f67c9c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c3124a640 1 -- 172.21.15.96:0/3299099714 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c240ab530 con 0x7f8c2c134e70 2024-09-13T17:42:26.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c3124a640 1 -- 172.21.15.96:0/3299099714 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c240a7050 con 0x7f8c2c134e70 2024-09-13T17:42:26.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c3124a640 1 -- 172.21.15.96:0/3299099714 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c240ab7e0 con 0x7f8c2c134e70 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- 172.21.15.96:0/3299099714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 msgr2=0x7f8c2c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 --2- 172.21.15.96:0/3299099714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c09e820 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009740 tx=0x7f8c2401c420 comp rx=0 tx=0).stop 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- 172.21.15.96:0/3299099714 shutdown_connections 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 --2- 172.21.15.96:0/3299099714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c09e820 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- 172.21.15.96:0/3299099714 >> 172.21.15.96:0/3299099714 conn(0x7f8c2c098a60 msgr2=0x7f8c2c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:26.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- 172.21.15.96:0/3299099714 shutdown_connections 2024-09-13T17:42:26.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- 172.21.15.96:0/3299099714 wait complete. 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 Processor -- start 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.889+0000 7f8c344b0640 1 -- start start 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c344b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c344b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c070770 con 0x7f8c2c134e70 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c3224c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c3224c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54698/0 (socket says 172.21.15.96:54698) 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c3224c640 1 -- 172.21.15.96:0/35287759 learned_addr learned my addr 172.21.15.96:0/35287759 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c3224c640 1 -- 172.21.15.96:0/35287759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c24009420 con 0x7f8c2c134e70 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c3224c640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c24019030 tx=0x7f8c2401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c240abde0 con 0x7f8c2c134e70 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c24004030 con 0x7f8c2c134e70 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c0a44a0 con 0x7f8c2c134e70 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c240042d0 con 0x7f8c2c134e70 2024-09-13T17:42:26.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c0a55c0 con 0x7f8c2c134e70 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c2c1368b0 con 0x7f8c2c134e70 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8c24003620 con 0x7f8c2c134e70 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 0x7f8c18037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8c240e2b80 con 0x7f8c2c134e70 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c31a4b640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 0x7f8c18037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:26.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.893+0000 7f8c31a4b640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 0x7f8c18037910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c10 tx=0x7f8c20001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:26.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:26.897+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c240b4580 con 0x7f8c2c134e70 2024-09-13T17:42:27.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.057+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c2c0a4630 con 0x7f8c2c134e70 2024-09-13T17:42:27.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c2affd640 1 -- 172.21.15.96:0/35287759 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f8c240b0020 con 0x7f8c2c134e70 2024-09-13T17:42:27.060 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:27.060 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:27.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 msgr2=0x7f8c18037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 0x7f8c18037910 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8c20000c10 tx=0x7f8c20001320 comp rx=0 tx=0).stop 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 msgr2=0x7f8c2c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c24019030 tx=0x7f8c2401ac60 comp rx=0 tx=0).stop 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 shutdown_connections 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c18035450 0x7f8c18037910 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 --2- 172.21.15.96:0/35287759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c2c134e70 0x7f8c2c06f6d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 >> 172.21.15.96:0/35287759 conn(0x7f8c2c098a60 msgr2=0x7f8c2c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 shutdown_connections 2024-09-13T17:42:27.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.061+0000 7f8c344b0640 1 -- 172.21.15.96:0/35287759 wait complete. 2024-09-13T17:42:27.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:27.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:27.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.201+0000 7f78a1fd4640 1 Processor -- start 2024-09-13T17:42:27.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.201+0000 7f78a1fd4640 1 -- start start 2024-09-13T17:42:27.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f789c09dc30 0x7f789c13ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789c1327b0 con 0x7f789c1329b0 2024-09-13T17:42:27.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c09bd90 con 0x7f789c09dc30 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f7892ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f789c09dc30 0x7f789c13ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f789c1329b0 0x7f789c132d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43726/0 (socket says 172.21.15.96:43726) 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789b7fe640 1 -- 172.21.15.96:0/3608429707 learned_addr learned my addr 172.21.15.96:0/3608429707 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789affd640 1 -- 172.21.15.96:0/3608429707 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181661841 0 0) 0x7f789c1327b0 con 0x7f789c1329b0 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789affd640 1 -- 172.21.15.96:0/3608429707 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7888003530 con 0x7f789c1329b0 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f7892ffd640 1 -- 172.21.15.96:0/3608429707 >> v1:172.21.15.96:6789/0 conn(0x7f789c1329b0 legacy=0x7f789c132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f7892ffd640 1 -- 172.21.15.96:0/3608429707 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c09b2a0 con 0x7f789c09dc30 2024-09-13T17:42:27.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f7892ffd640 1 --2- 172.21.15.96:0/3608429707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c13ad00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7884009740 tx=0x7f788401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f92ba69f5cee526 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789affd640 1 -- 172.21.15.96:0/3608429707 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78840ab530 con 0x7f789c09dc30 2024-09-13T17:42:27.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789affd640 1 -- 172.21.15.96:0/3608429707 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840a7050 con 0x7f789c09dc30 2024-09-13T17:42:27.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f789affd640 1 -- 172.21.15.96:0/3608429707 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f78840ab7e0 con 0x7f789c09dc30 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3608429707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 msgr2=0x7f789c13ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3608429707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c13ad00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7884009740 tx=0x7f788401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3608429707 shutdown_connections 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.205+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3608429707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c13ad00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3608429707 >> 172.21.15.96:0/3608429707 conn(0x7f789c06ef80 msgr2=0x7f789c0713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3608429707 shutdown_connections 2024-09-13T17:42:27.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3608429707 wait complete. 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 Processor -- start 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- start start 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0a2d30 con 0x7f789c09dc30 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f789b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f789b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54726/0 (socket says 172.21.15.96:54726) 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f789b7fe640 1 -- 172.21.15.96:0/3859865282 learned_addr learned my addr 172.21.15.96:0/3859865282 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f789b7fe640 1 -- 172.21.15.96:0/3859865282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884009420 con 0x7f789c09dc30 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f789b7fe640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c470 tx=0x7f788c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c005c20 con 0x7f789c09dc30 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c005d80 con 0x7f789c09dc30 2024-09-13T17:42:27.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c06f510 con 0x7f789c09dc30 2024-09-13T17:42:27.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f788c0071e0 con 0x7f789c09dc30 2024-09-13T17:42:27.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c0a12d0 con 0x7f789c09dc30 2024-09-13T17:42:27.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f788c007340 con 0x7f789c09dc30 2024-09-13T17:42:27.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c0a2e80 con 0x7f789c09dc30 2024-09-13T17:42:27.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 0x7f787c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f788c046a20 con 0x7f789c09dc30 2024-09-13T17:42:27.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7892ffd640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 0x7f787c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.209+0000 7f7892ffd640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 0x7f787c037910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f78840ad000 tx=0x7f7884008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.217+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f788c019300 con 0x7f789c09dc30 2024-09-13T17:42:27.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.361+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f789c0a0a70 con 0x7f789c09dc30 2024-09-13T17:42:27.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.361+0000 7f7898ff9640 1 -- 172.21.15.96:0/3859865282 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f788c017070 con 0x7f789c09dc30 2024-09-13T17:42:27.362 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:27.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 msgr2=0x7f787c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 0x7f787c037910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f78840ad000 tx=0x7f7884008040 comp rx=0 tx=0).stop 2024-09-13T17:42:27.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 msgr2=0x7f789c1cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f788c00c470 tx=0x7f788c00c930 comp rx=0 tx=0).stop 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 shutdown_connections 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f787c035450 0x7f787c037910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 --2- 172.21.15.96:0/3859865282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c09dc30 0x7f789c1cd500 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 >> 172.21.15.96:0/3859865282 conn(0x7f789c06ef80 msgr2=0x7f789c0706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 shutdown_connections 2024-09-13T17:42:27.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.365+0000 7f78a1fd4640 1 -- 172.21.15.96:0/3859865282 wait complete. 2024-09-13T17:42:27.380 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:27.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.501+0000 7f65d6b5a640 1 Processor -- start 2024-09-13T17:42:27.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.501+0000 7f65d6b5a640 1 -- start start 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.501+0000 7f65d6b5a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65d006f6d0 0x7f65d0134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.501+0000 7f65d6b5a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d006e130 con 0x7f65d009dc20 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.501+0000 7f65d6b5a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00a6750 con 0x7f65d006f6d0 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d48f6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65d006f6d0 0x7f65d0134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f65d009dc20 0x7f65d0134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43736/0 (socket says 172.21.15.96:43736) 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d48f6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65d006f6d0 0x7f65d0134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54728/0 (socket says 172.21.15.96:54728) 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cffff640 1 -- 172.21.15.96:0/1223711443 learned_addr learned my addr 172.21.15.96:0/1223711443 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d48f6640 1 -- 172.21.15.96:0/1223711443 >> v1:172.21.15.96:6789/0 conn(0x7f65d009dc20 legacy=0x7f65d0134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d48f6640 1 -- 172.21.15.96:0/1223711443 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00712e0 con 0x7f65d006f6d0 2024-09-13T17:42:27.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cf7fe640 1 -- 172.21.15.96:0/1223711443 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325915677 0 0) 0x7f65d006e130 con 0x7f65d009dc20 2024-09-13T17:42:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d48f6640 1 --2- 172.21.15.96:0/1223711443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d0134ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009740 tx=0x7f65c001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01d970b4cdbf1ca server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cf7fe640 1 -- 172.21.15.96:0/1223711443 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c00ab400 con 0x7f65d006f6d0 2024-09-13T17:42:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cf7fe640 1 -- 172.21.15.96:0/1223711443 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00a7050 con 0x7f65d006f6d0 2024-09-13T17:42:27.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65cf7fe640 1 -- 172.21.15.96:0/1223711443 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c00ab6b0 con 0x7f65d006f6d0 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 -- 172.21.15.96:0/1223711443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 msgr2=0x7f65d0134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/1223711443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d0134ab0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009740 tx=0x7f65c001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 -- 172.21.15.96:0/1223711443 shutdown_connections 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/1223711443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d0134ab0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 -- 172.21.15.96:0/1223711443 >> 172.21.15.96:0/1223711443 conn(0x7f65d0098a60 msgr2=0x7f65d009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 -- 172.21.15.96:0/1223711443 shutdown_connections 2024-09-13T17:42:27.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.505+0000 7f65d6b5a640 1 -- 172.21.15.96:0/1223711443 wait complete. 2024-09-13T17:42:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 Processor -- start 2024-09-13T17:42:27.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 -- start start 2024-09-13T17:42:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00a0a00 con 0x7f65d006f6d0 2024-09-13T17:42:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d48f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d48f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54730/0 (socket says 172.21.15.96:54730) 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d48f6640 1 -- 172.21.15.96:0/3400088383 learned_addr learned my addr 172.21.15.96:0/3400088383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d48f6640 1 -- 172.21.15.96:0/3400088383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009420 con 0x7f65d006f6d0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d48f6640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65c0019fc0 tx=0x7f65c001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c00ab400 con 0x7f65d006f6d0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00ab560 con 0x7f65d006f6d0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65c00ab820 con 0x7f65d006f6d0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00a02f0 con 0x7f65d006f6d0 2024-09-13T17:42:27.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d009aac0 con 0x7f65d006f6d0 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d0132d90 con 0x7f65d006f6d0 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f65c00ab980 con 0x7f65d006f6d0 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 0x7f65bc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f65c00e3630 con 0x7f65d006f6d0 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cffff640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 0x7f65bc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.509+0000 7f65cffff640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 0x7f65bc037910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65c4007fd0 tx=0x7f65c4007940 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.513+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65c00c4db0 con 0x7f65d006f6d0 2024-09-13T17:42:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65d0134ab0 con 0x7f65d006f6d0 2024-09-13T17:42:27.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65cd7fa640 1 -- 172.21.15.96:0/3400088383 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65c00b0070 con 0x7f65d006f6d0 2024-09-13T17:42:27.673 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 msgr2=0x7f65bc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 0x7f65bc037910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f65c4007fd0 tx=0x7f65c4007940 comp rx=0 tx=0).stop 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 msgr2=0x7f65d00a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f65c0019fc0 tx=0x7f65c001abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 shutdown_connections 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f65bc035450 0x7f65bc037910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 --2- 172.21.15.96:0/3400088383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65d006f6d0 0x7f65d00a1150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 >> 172.21.15.96:0/3400088383 conn(0x7f65d0098a60 msgr2=0x7f65d0131b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 shutdown_connections 2024-09-13T17:42:27.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.673+0000 7f65d6b5a640 1 -- 172.21.15.96:0/3400088383 wait complete. 2024-09-13T17:42:27.692 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.813+0000 7fa872bfb640 1 Processor -- start 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.813+0000 7fa872bfb640 1 -- start start 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa86c137d60 0x7fa86c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa86c0a00b0 con 0x7fa86c138140 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86c0a12d0 con 0x7fa86c137d60 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa870997640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa86c137d60 0x7fa86c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa86bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa86c138140 0x7fa86c138590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43742/0 (socket says 172.21.15.96:43742) 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa86bfff640 1 -- 172.21.15.96:0/1448461487 learned_addr learned my addr 172.21.15.96:0/1448461487 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa870997640 1 -- 172.21.15.96:0/1448461487 >> v1:172.21.15.96:6789/0 conn(0x7fa86c138140 legacy=0x7fa86c138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa870997640 1 -- 172.21.15.96:0/1448461487 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c0a0350 con 0x7fa86c137d60 2024-09-13T17:42:27.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa870997640 1 --2- 172.21.15.96:0/1448461487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a0d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009740 tx=0x7fa85c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dc2a3aaf40d098 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa86b7fe640 1 -- 172.21.15.96:0/1448461487 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa85c0ab530 con 0x7fa86c137d60 2024-09-13T17:42:27.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa86b7fe640 1 -- 172.21.15.96:0/1448461487 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c0a7050 con 0x7fa86c137d60 2024-09-13T17:42:27.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa86b7fe640 1 -- 172.21.15.96:0/1448461487 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa85c0ab7e0 con 0x7fa86c137d60 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- 172.21.15.96:0/1448461487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 msgr2=0x7fa86c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 --2- 172.21.15.96:0/1448461487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a0d90 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa85c009740 tx=0x7fa85c01c390 comp rx=0 tx=0).stop 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- 172.21.15.96:0/1448461487 shutdown_connections 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 --2- 172.21.15.96:0/1448461487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a0d90 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- 172.21.15.96:0/1448461487 >> 172.21.15.96:0/1448461487 conn(0x7fa86c06ef80 msgr2=0x7fa86c0713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- 172.21.15.96:0/1448461487 shutdown_connections 2024-09-13T17:42:27.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.817+0000 7fa872bfb640 1 -- 172.21.15.96:0/1448461487 wait complete. 2024-09-13T17:42:27.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 Processor -- start 2024-09-13T17:42:27.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 -- start start 2024-09-13T17:42:27.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c0a5970 con 0x7fa86c137d60 2024-09-13T17:42:27.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa870997640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa870997640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54738/0 (socket says 172.21.15.96:54738) 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa870997640 1 -- 172.21.15.96:0/2229302783 learned_addr learned my addr 172.21.15.96:0/2229302783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa870997640 1 -- 172.21.15.96:0/2229302783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c009420 con 0x7fa86c137d60 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa870997640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ae040 tx=0x7fa85c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa85c0b7660 con 0x7fa86c137d60 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85c0b77c0 con 0x7fa86c137d60 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c0a44b0 con 0x7fa86c137d60 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa85c0b7a60 con 0x7fa86c137d60 2024-09-13T17:42:27.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c0a2e90 con 0x7fa86c137d60 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c0a46e0 con 0x7fa86c137d60 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fa85c0b7bc0 con 0x7fa86c137d60 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 0x7fa8580378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fa85c0e24a0 con 0x7fa86c137d60 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa86bfff640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 0x7fa8580378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:27.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.821+0000 7fa86bfff640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 0x7fa8580378c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa860000f70 tx=0x7fa860007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:27.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.825+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa85c0c7cd0 con 0x7fa86c137d60 2024-09-13T17:42:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.969+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa86c0a0d90 con 0x7fa86c137d60 2024-09-13T17:42:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa8697fa640 1 -- 172.21.15.96:0/2229302783 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa85c0b00a0 con 0x7fa86c137d60 2024-09-13T17:42:27.972 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 msgr2=0x7fa8580378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 0x7fa8580378c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa860000f70 tx=0x7fa860007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 msgr2=0x7fa86c0a5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fa85c0ae040 tx=0x7fa85c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 shutdown_connections 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fa858035400 0x7fa8580378c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 --2- 172.21.15.96:0/2229302783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa86c137d60 0x7fa86c0a5430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 >> 172.21.15.96:0/2229302783 conn(0x7fa86c06ef80 msgr2=0x7fa86c1324a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 shutdown_connections 2024-09-13T17:42:27.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:27.973+0000 7fa872bfb640 1 -- 172.21.15.96:0/2229302783 wait complete. 2024-09-13T17:42:27.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:28.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.105+0000 7f8c63404640 1 Processor -- start 2024-09-13T17:42:28.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.105+0000 7f8c63404640 1 -- start start 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c63404640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c5c09ef30 0x7f8c5c13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c63404640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c0a79f0 con 0x7f8c5c136090 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c63404640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c5c06a650 con 0x7f8c5c09ef30 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c6099f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8c5c09ef30 0x7f8c5c13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c611a0640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8c5c136090 0x7f8c5c136470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43756/0 (socket says 172.21.15.96:43756) 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c611a0640 1 -- 172.21.15.96:0/2193733567 learned_addr learned my addr 172.21.15.96:0/2193733567 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:28.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c6099f640 1 -- 172.21.15.96:0/2193733567 >> v1:172.21.15.96:6789/0 conn(0x7f8c5c136090 legacy=0x7f8c5c136470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c6099f640 1 -- 172.21.15.96:0/2193733567 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c09c3d0 con 0x7f8c5c09ef30 2024-09-13T17:42:28.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c6099f640 1 --2- 172.21.15.96:0/2193733567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c13aff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009740 tx=0x7f8c5001c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539b4d2fa364fa96 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c4ffff640 1 -- 172.21.15.96:0/2193733567 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c500ab530 con 0x7f8c5c09ef30 2024-09-13T17:42:28.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c4ffff640 1 -- 172.21.15.96:0/2193733567 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c500a7050 con 0x7f8c5c09ef30 2024-09-13T17:42:28.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.109+0000 7f8c4ffff640 1 -- 172.21.15.96:0/2193733567 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c500ab7e0 con 0x7f8c5c09ef30 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/2193733567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 msgr2=0x7f8c5c13aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 --2- 172.21.15.96:0/2193733567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c13aff0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009740 tx=0x7f8c5001c220 comp rx=0 tx=0).stop 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/2193733567 shutdown_connections 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 --2- 172.21.15.96:0/2193733567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c13aff0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/2193733567 >> 172.21.15.96:0/2193733567 conn(0x7f8c5c06e020 msgr2=0x7f8c5c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/2193733567 shutdown_connections 2024-09-13T17:42:28.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/2193733567 wait complete. 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 Processor -- start 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- start start 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0a60d0 con 0x7f8c5c09ef30 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c611a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c611a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54760/0 (socket says 172.21.15.96:54760) 2024-09-13T17:42:28.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c611a0640 1 -- 172.21.15.96:0/792453001 learned_addr learned my addr 172.21.15.96:0/792453001 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c611a0640 1 -- 172.21.15.96:0/792453001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c50009420 con 0x7f8c5c09ef30 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c611a0640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800c700 tx=0x7f8c5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c58007200 con 0x7f8c5c09ef30 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58007380 con 0x7f8c5c09ef30 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c0a6300 con 0x7f8c5c09ef30 2024-09-13T17:42:28.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c58007660 con 0x7f8c5c09ef30 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c13ac30 con 0x7f8c5c09ef30 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8c580047f0 con 0x7f8c5c09ef30 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 0x7f8c40037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8c58046e20 con 0x7f8c5c09ef30 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c13a790 con 0x7f8c5c09ef30 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.113+0000 7f8c6099f640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 0x7f8c40037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.117+0000 7f8c6099f640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 0x7f8c40037910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ad000 tx=0x7f8c50008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.121+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8c58018030 con 0x7f8c5c09ef30 2024-09-13T17:42:28.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.281+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c5c073a50 con 0x7f8c5c09ef30 2024-09-13T17:42:28.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.281+0000 7f8c4dffb640 1 -- 172.21.15.96:0/792453001 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f8c58018020 con 0x7f8c5c09ef30 2024-09-13T17:42:28.284 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:28.284 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 msgr2=0x7f8c40037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 0x7f8c40037910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8c500ad000 tx=0x7f8c50008040 comp rx=0 tx=0).stop 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 msgr2=0x7f8c5c098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8c5800c700 tx=0x7f8c5800cbc0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 shutdown_connections 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8c40035450 0x7f8c40037910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 --2- 172.21.15.96:0/792453001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c5c09ef30 0x7f8c5c098a60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 >> 172.21.15.96:0/792453001 conn(0x7f8c5c06e020 msgr2=0x7f8c5c135260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 shutdown_connections 2024-09-13T17:42:28.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.285+0000 7f8c63404640 1 -- 172.21.15.96:0/792453001 wait complete. 2024-09-13T17:42:28.302 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:28.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:28.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98ad53b640 1 Processor -- start 2024-09-13T17:42:28.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98ad53b640 1 -- start start 2024-09-13T17:42:28.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98ad53b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f98a809d690 0x7f98a8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98ad53b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a8138050 con 0x7f98a81387d0 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98ad53b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a809c410 con 0x7f98a809d690 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f98a809d690 0x7f98a8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a67fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f98a81387d0 0x7f98a8134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43766/0 (socket says 172.21.15.96:43766) 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a67fc640 1 -- 172.21.15.96:0/1128753940 learned_addr learned my addr 172.21.15.96:0/1128753940 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a6ffd640 1 -- 172.21.15.96:0/1128753940 >> v1:172.21.15.96:6789/0 conn(0x7f98a81387d0 legacy=0x7f98a8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a6ffd640 1 -- 172.21.15.96:0/1128753940 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a809cba0 con 0x7f98a809d690 2024-09-13T17:42:28.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.425+0000 7f98a5ffb640 1 -- 172.21.15.96:0/1128753940 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805563667 0 0) 0x7f989c001120 con 0x7f98a81387d0 2024-09-13T17:42:28.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 --2- 172.21.15.96:0/1128753940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a8138290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9898009740 tx=0x7f989801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b274cb6b5db589 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a5ffb640 1 -- 172.21.15.96:0/1128753940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98980a8840 con 0x7f98a809d690 2024-09-13T17:42:28.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a5ffb640 1 -- 172.21.15.96:0/1128753940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98980a89a0 con 0x7f98a809d690 2024-09-13T17:42:28.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a5ffb640 1 -- 172.21.15.96:0/1128753940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98980a8c60 con 0x7f98a809d690 2024-09-13T17:42:28.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/1128753940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 msgr2=0x7f98a8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 --2- 172.21.15.96:0/1128753940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a8138290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9898009740 tx=0x7f989801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:28.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/1128753940 shutdown_connections 2024-09-13T17:42:28.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 --2- 172.21.15.96:0/1128753940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a8138290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/1128753940 >> 172.21.15.96:0/1128753940 conn(0x7f98a806bc60 msgr2=0x7f98a806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/1128753940 shutdown_connections 2024-09-13T17:42:28.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/1128753940 wait complete. 2024-09-13T17:42:28.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 Processor -- start 2024-09-13T17:42:28.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- start start 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a80aab40 con 0x7f98a809d690 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54776/0 (socket says 172.21.15.96:54776) 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 -- 172.21.15.96:0/2102346272 learned_addr learned my addr 172.21.15.96:0/2102346272 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 -- 172.21.15.96:0/2102346272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9898009420 con 0x7f98a809d690 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98a6ffd640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f989801c840 tx=0x7f989801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98980b0840 con 0x7f98a809d690 2024-09-13T17:42:28.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98980b09a0 con 0x7f98a809d690 2024-09-13T17:42:28.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98980b0c60 con 0x7f98a809d690 2024-09-13T17:42:28.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a80a9590 con 0x7f98a809d690 2024-09-13T17:42:28.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.429+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a80a8ac0 con 0x7f98a809d690 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a809bc00 con 0x7f98a809d690 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f98980b7040 con 0x7f98a809d690 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f988f7fe640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 0x7f9894037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f98980e5410 con 0x7f98a809d690 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f98a67fc640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 0x7f9894037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.433+0000 7f98a67fc640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 0x7f9894037910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c10 tx=0x7f989c001260 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.437+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98a809bc00 con 0x7f98a809d690 2024-09-13T17:42:28.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.589+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98a80a93e0 con 0x7f98a809d690 2024-09-13T17:42:28.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.589+0000 7f988f7fe640 1 -- 172.21.15.96:0/2102346272 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98980b50a0 con 0x7f98a809d690 2024-09-13T17:42:28.592 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 msgr2=0x7f9894037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 0x7f9894037910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f989c000c10 tx=0x7f989c001260 comp rx=0 tx=0).stop 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 msgr2=0x7f98a80aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f989801c840 tx=0x7f989801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 shutdown_connections 2024-09-13T17:42:28.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9894035450 0x7f9894037910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 --2- 172.21.15.96:0/2102346272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98a809d690 0x7f98a80aad20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 >> 172.21.15.96:0/2102346272 conn(0x7f98a806bc60 msgr2=0x7f98a8099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 shutdown_connections 2024-09-13T17:42:28.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.593+0000 7f98ad53b640 1 -- 172.21.15.96:0/2102346272 wait complete. 2024-09-13T17:42:28.610 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:28.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8457e640 1 Processor -- start 2024-09-13T17:42:28.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8457e640 1 -- start start 2024-09-13T17:42:28.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8457e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f7c09e820 0x7f3f7c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8457e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7c0a68b0 con 0x7f3f7c134e70 2024-09-13T17:42:28.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8457e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7c09ddc0 con 0x7f3f7c09e820 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81b19640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f7c09e820 0x7f3f7c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8231a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3f7c134e70 0x7f3f7c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43774/0 (socket says 172.21.15.96:43774) 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f8231a640 1 -- 172.21.15.96:0/1114008848 learned_addr learned my addr 172.21.15.96:0/1114008848 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81b19640 1 -- 172.21.15.96:0/1114008848 >> v1:172.21.15.96:6789/0 conn(0x7f3f7c134e70 legacy=0x7f3f7c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81b19640 1 -- 172.21.15.96:0/1114008848 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c136970 con 0x7f3f7c09e820 2024-09-13T17:42:28.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81318640 1 -- 172.21.15.96:0/1114008848 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847082928 0 0) 0x7f3f7c0a68b0 con 0x7f3f7c134e70 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81b19640 1 --2- 172.21.15.96:0/1114008848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c13ae50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009740 tx=0x7f3f6c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d177d66ce27c21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81318640 1 -- 172.21.15.96:0/1114008848 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f6c0ab530 con 0x7f3f7c09e820 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81318640 1 -- 172.21.15.96:0/1114008848 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0a7050 con 0x7f3f7c09e820 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.733+0000 7f3f81318640 1 -- 172.21.15.96:0/1114008848 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f6c0ab7e0 con 0x7f3f7c09e820 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/1114008848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 msgr2=0x7f3f7c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 --2- 172.21.15.96:0/1114008848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c13ae50 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009740 tx=0x7f3f6c01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/1114008848 shutdown_connections 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 --2- 172.21.15.96:0/1114008848 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c13ae50 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/1114008848 >> 172.21.15.96:0/1114008848 conn(0x7f3f7c098a60 msgr2=0x7f3f7c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/1114008848 shutdown_connections 2024-09-13T17:42:28.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/1114008848 wait complete. 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 Processor -- start 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- start start 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c13a490 con 0x7f3f7c09e820 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8231a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8231a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54786/0 (socket says 172.21.15.96:54786) 2024-09-13T17:42:28.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8231a640 1 -- 172.21.15.96:0/3450508007 learned_addr learned my addr 172.21.15.96:0/3450508007 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8231a640 1 -- 172.21.15.96:0/3450508007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c009420 con 0x7f3f7c09e820 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8231a640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c420 tx=0x7f3f7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f780074d0 con 0x7f3f7c09e820 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78007630 con 0x7f3f7c09e820 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f78004960 con 0x7f3f7c09e820 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c0a4060 con 0x7f3f7c09e820 2024-09-13T17:42:28.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c0a61a0 con 0x7f3f7c09e820 2024-09-13T17:42:28.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f3f78004ac0 con 0x7f3f7c09e820 2024-09-13T17:42:28.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c0a1e10 con 0x7f3f7c09e820 2024-09-13T17:42:28.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 0x7f3f600378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:28.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f3f78046cb0 con 0x7f3f7c09e820 2024-09-13T17:42:28.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f81b19640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 0x7f3f600378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:28.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.737+0000 7f3f81b19640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 0x7f3f600378c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ad000 tx=0x7f3f6c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:28.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.741+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f7801acd0 con 0x7f3f7c09e820 2024-09-13T17:42:28.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f7c06a1b0 con 0x7f3f7c09e820 2024-09-13T17:42:28.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f72ffd640 1 -- 172.21.15.96:0/3450508007 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3f78018030 con 0x7f3f7c09e820 2024-09-13T17:42:28.890 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:28.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 msgr2=0x7f3f600378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 0x7f3f600378c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ad000 tx=0x7f3f6c008040 comp rx=0 tx=0).stop 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 msgr2=0x7f3f7c0a42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c420 tx=0x7f3f7800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 shutdown_connections 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f3f60035400 0x7f3f600378c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 --2- 172.21.15.96:0/3450508007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f7c09e820 0x7f3f7c0a42b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 >> 172.21.15.96:0/3450508007 conn(0x7f3f7c098a60 msgr2=0x7f3f7c133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 shutdown_connections 2024-09-13T17:42:28.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:28.889+0000 7f3f8457e640 1 -- 172.21.15.96:0/3450508007 wait complete. 2024-09-13T17:42:28.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:29.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d9dd8a640 1 Processor -- start 2024-09-13T17:42:29.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d9dd8a640 1 -- start start 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d9dd8a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9d9809e820 0x7f9d9813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d9dd8a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d980a68b0 con 0x7f9d98134e70 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d9dd8a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9809ddc0 con 0x7f9d9809e820 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d977fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9d98134e70 0x7f9d98135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43776/0 (socket says 172.21.15.96:43776) 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d977fe640 1 -- 172.21.15.96:0/2754505817 learned_addr learned my addr 172.21.15.96:0/2754505817 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.033+0000 7f9d96ffd640 1 --2- 172.21.15.96:0/2754505817 >> v2:172.21.15.96:3300/0 conn(0x7f9d9809e820 0x7f9d9813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d967fc640 1 -- 172.21.15.96:0/2754505817 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383976958 0 0) 0x7f9d980a68b0 con 0x7f9d98134e70 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d967fc640 1 -- 172.21.15.96:0/2754505817 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d80003530 con 0x7f9d98134e70 2024-09-13T17:42:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d96ffd640 1 -- 172.21.15.96:0/2754505817 >> v1:172.21.15.96:6789/0 conn(0x7f9d98134e70 legacy=0x7f9d98135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d96ffd640 1 -- 172.21.15.96:0/2754505817 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98136970 con 0x7f9d9809e820 2024-09-13T17:42:29.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d96ffd640 1 --2- 172.21.15.96:0/2754505817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9813ae50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d90 tx=0x7f9d8c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b00d4919b18342 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d967fc640 1 -- 172.21.15.96:0/2754505817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d8c0a57b0 con 0x7f9d9809e820 2024-09-13T17:42:29.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d967fc640 1 -- 172.21.15.96:0/2754505817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c0a5910 con 0x7f9d9809e820 2024-09-13T17:42:29.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d967fc640 1 -- 172.21.15.96:0/2754505817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d8c0a5bd0 con 0x7f9d9809e820 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/2754505817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 msgr2=0x7f9d9813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/2754505817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9813ae50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009d90 tx=0x7f9d8c01c970 comp rx=0 tx=0).stop 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/2754505817 shutdown_connections 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/2754505817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9813ae50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/2754505817 >> 172.21.15.96:0/2754505817 conn(0x7f9d98098a60 msgr2=0x7f9d9809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/2754505817 shutdown_connections 2024-09-13T17:42:29.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.037+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/2754505817 wait complete. 2024-09-13T17:42:29.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 Processor -- start 2024-09-13T17:42:29.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 -- start start 2024-09-13T17:42:29.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d98071560 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54802/0 (socket says 172.21.15.96:54802) 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d977fe640 1 -- 172.21.15.96:0/65970030 learned_addr learned my addr 172.21.15.96:0/65970030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d977fe640 1 -- 172.21.15.96:0/65970030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c009a70 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d977fe640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c450 tx=0x7f9d9000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d90005c20 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90005d80 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9811e3b0 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d900071a0 con 0x7f9d9809e820 2024-09-13T17:42:29.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d98070850 con 0x7f9d9809e820 2024-09-13T17:42:29.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d98070ba0 con 0x7f9d9809e820 2024-09-13T17:42:29.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f9d90007300 con 0x7f9d9809e820 2024-09-13T17:42:29.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 0x7f9d84037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9d90046b80 con 0x7f9d9809e820 2024-09-13T17:42:29.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d96ffd640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 0x7f9d84037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.041+0000 7f9d96ffd640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 0x7f9d84037910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0af000 tx=0x7f9d8c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.045+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9d9001b3b0 con 0x7f9d9809e820 2024-09-13T17:42:29.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.201+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d9813a420 con 0x7f9d9809e820 2024-09-13T17:42:29.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.201+0000 7f9d7ffff640 1 -- 172.21.15.96:0/65970030 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d90017070 con 0x7f9d9809e820 2024-09-13T17:42:29.204 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:29.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 msgr2=0x7f9d84037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 0x7f9d84037910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0af000 tx=0x7f9d8c0a7070 comp rx=0 tx=0).stop 2024-09-13T17:42:29.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 msgr2=0x7f9d9806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000c450 tx=0x7f9d9000c910 comp rx=0 tx=0).stop 2024-09-13T17:42:29.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 shutdown_connections 2024-09-13T17:42:29.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9d84035450 0x7f9d84037910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 --2- 172.21.15.96:0/65970030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d9809e820 0x7f9d9806f6d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 >> 172.21.15.96:0/65970030 conn(0x7f9d98098a60 msgr2=0x7f9d98133e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 shutdown_connections 2024-09-13T17:42:29.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.205+0000 7f9d9dd8a640 1 -- 172.21.15.96:0/65970030 wait complete. 2024-09-13T17:42:29.243 INFO:tasks.cephfs_test_runner:Starting test: test_dump (tasks.cephfs.test_fstop.TestFSTop) Tests 'cephfs-top --dump' output is valid 2024-09-13T17:42:29.244 INFO:tasks.cephfs_test_runner:test_dump (tasks.cephfs.test_fstop.TestFSTop) 2024-09-13T17:42:29.245 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:29.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51b4e2c640 1 Processor -- start 2024-09-13T17:42:29.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51b4e2c640 1 -- start start 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51b4e2c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f51b0135270 0x7f51b013cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51b4e2c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b0070de0 con 0x7f51b0134e90 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51b4e2c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b0071d60 con 0x7f51b0135270 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51add9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f51b0135270 0x7f51b013cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51ae59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f51b0134e90 0x7f51b0071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43786/0 (socket says 172.21.15.96:43786) 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51add9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f51b0135270 0x7f51b013cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54810/0 (socket says 172.21.15.96:54810) 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.361+0000 7f51ae59c640 1 -- 172.21.15.96:0/2217549063 learned_addr learned my addr 172.21.15.96:0/2217549063 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:29.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51add9b640 1 -- 172.21.15.96:0/2217549063 >> v1:172.21.15.96:6789/0 conn(0x7f51b0134e90 legacy=0x7f51b0071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51add9b640 1 -- 172.21.15.96:0/2217549063 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b006f110 con 0x7f51b0135270 2024-09-13T17:42:29.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51add9b640 1 --2- 172.21.15.96:0/2217549063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0135270 0x7f51b013cfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009740 tx=0x7f51a001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13343081c1e8e0ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51ad59a640 1 -- 172.21.15.96:0/2217549063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a00ab530 con 0x7f51b0135270 2024-09-13T17:42:29.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51ad59a640 1 -- 172.21.15.96:0/2217549063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00a7050 con 0x7f51b0135270 2024-09-13T17:42:29.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51ad59a640 1 -- 172.21.15.96:0/2217549063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51a00ab7e0 con 0x7f51b0135270 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 -- 172.21.15.96:0/2217549063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0135270 msgr2=0x7f51b013cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/2217549063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0135270 0x7f51b013cfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009740 tx=0x7f51a001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 -- 172.21.15.96:0/2217549063 shutdown_connections 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/2217549063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0135270 0x7f51b013cfe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 -- 172.21.15.96:0/2217549063 >> 172.21.15.96:0/2217549063 conn(0x7f51b0098a60 msgr2=0x7f51b009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 -- 172.21.15.96:0/2217549063 shutdown_connections 2024-09-13T17:42:29.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.365+0000 7f51b4e2c640 1 -- 172.21.15.96:0/2217549063 wait complete. 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 Processor -- start 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 -- start start 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0136c80 con 0x7f51b0134e90 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51ae59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54820/0 (socket says 172.21.15.96:54820) 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51ae59c640 1 -- 172.21.15.96:0/1926999801 learned_addr learned my addr 172.21.15.96:0/1926999801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:29.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51ae59c640 1 -- 172.21.15.96:0/1926999801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009420 con 0x7f51b0134e90 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51ae59c640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f519800c3a0 tx=0x7f519800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5198005c20 con 0x7f51b0134e90 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5198005d80 con 0x7f51b0134e90 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b011d5f0 con 0x7f51b0134e90 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5198007200 con 0x7f51b0134e90 2024-09-13T17:42:29.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b00a5f30 con 0x7f51b0134e90 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5198007360 con 0x7f51b0134e90 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b011d2e0 con 0x7f51b0134e90 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 0x7f5190037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5198046b60 con 0x7f51b0134e90 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51add9b640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 0x7f5190037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.369+0000 7f51add9b640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 0x7f5190037910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f51a00ad000 tx=0x7f51a00a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.373+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5198012400 con 0x7f51b0134e90 2024-09-13T17:42:29.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51b0071530 con 0x7f51b0134e90 2024-09-13T17:42:29.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51a6ffd640 1 -- 172.21.15.96:0/1926999801 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f5198017020 con 0x7f51b0134e90 2024-09-13T17:42:29.521 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:29.521 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:29.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 msgr2=0x7f5190037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 0x7f5190037910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f51a00ad000 tx=0x7f51a00a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:29.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 msgr2=0x7f51b011e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f519800c3a0 tx=0x7f519800c860 comp rx=0 tx=0).stop 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 shutdown_connections 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5190035450 0x7f5190037910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 --2- 172.21.15.96:0/1926999801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51b0134e90 0x7f51b011e030 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 >> 172.21.15.96:0/1926999801 conn(0x7f51b0098a60 msgr2=0x7f51b009add0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 shutdown_connections 2024-09-13T17:42:29.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.521+0000 7f51b4e2c640 1 -- 172.21.15.96:0/1926999801 wait complete. 2024-09-13T17:42:29.538 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:29.539 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:29.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.657+0000 7f8178212640 1 Processor -- start 2024-09-13T17:42:29.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.657+0000 7f8178212640 1 -- start start 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f817009e820 0x7f817013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700a68b0 con 0x7f8170134e70 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f817009ddc0 con 0x7f817009e820 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8175fae640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8170134e70 0x7f8170135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43796/0 (socket says 172.21.15.96:43796) 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f81757ad640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f817009e820 0x7f817013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8175fae640 1 -- 172.21.15.96:0/1233471106 learned_addr learned my addr 172.21.15.96:0/1233471106 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:29.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f81757ad640 1 -- 172.21.15.96:0/1233471106 >> v1:172.21.15.96:6789/0 conn(0x7f8170134e70 legacy=0x7f8170135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f81757ad640 1 -- 172.21.15.96:0/1233471106 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8170136970 con 0x7f817009e820 2024-09-13T17:42:29.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f81757ad640 1 --2- 172.21.15.96:0/1233471106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817013ae50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8160009740 tx=0x7f816001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b5d2e2c77c26e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8174fac640 1 -- 172.21.15.96:0/1233471106 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81600ab530 con 0x7f817009e820 2024-09-13T17:42:29.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8174fac640 1 -- 172.21.15.96:0/1233471106 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81600a7050 con 0x7f817009e820 2024-09-13T17:42:29.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8174fac640 1 -- 172.21.15.96:0/1233471106 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81600ab7e0 con 0x7f817009e820 2024-09-13T17:42:29.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- 172.21.15.96:0/1233471106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 msgr2=0x7f817013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 --2- 172.21.15.96:0/1233471106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817013ae50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8160009740 tx=0x7f816001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:29.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- 172.21.15.96:0/1233471106 shutdown_connections 2024-09-13T17:42:29.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 --2- 172.21.15.96:0/1233471106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817013ae50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- 172.21.15.96:0/1233471106 >> 172.21.15.96:0/1233471106 conn(0x7f8170098a60 msgr2=0x7f817009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- 172.21.15.96:0/1233471106 shutdown_connections 2024-09-13T17:42:29.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.661+0000 7f8178212640 1 -- 172.21.15.96:0/1233471106 wait complete. 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 Processor -- start 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 -- start start 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817013a650 con 0x7f817009e820 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8175fae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8175fae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54840/0 (socket says 172.21.15.96:54840) 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8175fae640 1 -- 172.21.15.96:0/4152722389 learned_addr learned my addr 172.21.15.96:0/4152722389 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:29.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8175fae640 1 -- 172.21.15.96:0/4152722389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160009420 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8175fae640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f816800cb20 tx=0x7f816800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f816800abf0 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816800ad50 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f817011c930 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8168012400 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170071610 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8168012560 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817011cb30 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 0x7f8154037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f81680476a0 con 0x7f817009e820 2024-09-13T17:42:29.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81757ad640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 0x7f8154037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.665+0000 7f81757ad640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 0x7f8154037910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81600ad000 tx=0x7f8160008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.669+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8168007070 con 0x7f817009e820 2024-09-13T17:42:29.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.825+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f817006a1b0 con 0x7f817009e820 2024-09-13T17:42:29.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.825+0000 7f81667fc640 1 -- 172.21.15.96:0/4152722389 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8168017070 con 0x7f817009e820 2024-09-13T17:42:29.827 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 msgr2=0x7f8154037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 0x7f8154037910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81600ad000 tx=0x7f8160008040 comp rx=0 tx=0).stop 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 msgr2=0x7f817012ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f816800cb20 tx=0x7f816800a060 comp rx=0 tx=0).stop 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 shutdown_connections 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8154035450 0x7f8154037910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 --2- 172.21.15.96:0/4152722389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f817009e820 0x7f817012ef70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 >> 172.21.15.96:0/4152722389 conn(0x7f8170098a60 msgr2=0x7f8170134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 shutdown_connections 2024-09-13T17:42:29.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.829+0000 7f8178212640 1 -- 172.21.15.96:0/4152722389 wait complete. 2024-09-13T17:42:29.846 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:29.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcf80e640 1 Processor -- start 2024-09-13T17:42:29.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcf80e640 1 -- start start 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcf80e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5fc806f6d0 0x7f5fc8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcf80e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc806e130 con 0x7f5fc809dc20 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcf80e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc80a6750 con 0x7f5fc806f6d0 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcd5aa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5fc806f6d0 0x7f5fc8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcd5aa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5fc806f6d0 0x7f5fc8134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54844/0 (socket says 172.21.15.96:54844) 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fcd5aa640 1 -- 172.21.15.96:0/3206572692 learned_addr learned my addr 172.21.15.96:0/3206572692 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.969+0000 7f5fccda9640 1 --1- 172.21.15.96:0/3206572692 >> v1:172.21.15.96:6789/0 conn(0x7f5fc809dc20 0x7f5fc8134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43802/0 (socket says 172.21.15.96:43802) 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcd5aa640 1 -- 172.21.15.96:0/3206572692 >> v1:172.21.15.96:6789/0 conn(0x7f5fc809dc20 legacy=0x7f5fc8134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcd5aa640 1 -- 172.21.15.96:0/3206572692 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc80712e0 con 0x7f5fc806f6d0 2024-09-13T17:42:29.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcd5aa640 1 --2- 172.21.15.96:0/3206572692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc806f6d0 0x7f5fc8134ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097f0 tx=0x7f5fc401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74459e813c3fe2b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fbffff640 1 -- 172.21.15.96:0/3206572692 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fc40ab400 con 0x7f5fc806f6d0 2024-09-13T17:42:29.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fbffff640 1 -- 172.21.15.96:0/3206572692 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40a7050 con 0x7f5fc806f6d0 2024-09-13T17:42:29.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fbffff640 1 -- 172.21.15.96:0/3206572692 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fc40ab6b0 con 0x7f5fc806f6d0 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3206572692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc806f6d0 msgr2=0x7f5fc8134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3206572692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc806f6d0 0x7f5fc8134ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097f0 tx=0x7f5fc401c450 comp rx=0 tx=0).stop 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3206572692 shutdown_connections 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3206572692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc806f6d0 0x7f5fc8134ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3206572692 >> 172.21.15.96:0/3206572692 conn(0x7f5fc8098a60 msgr2=0x7f5fc809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3206572692 shutdown_connections 2024-09-13T17:42:29.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.973+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3206572692 wait complete. 2024-09-13T17:42:29.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 Processor -- start 2024-09-13T17:42:29.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 -- start start 2024-09-13T17:42:29.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc81394d0 con 0x7f5fc809dc20 2024-09-13T17:42:29.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcd5aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcd5aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54846/0 (socket says 172.21.15.96:54846) 2024-09-13T17:42:29.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcd5aa640 1 -- 172.21.15.96:0/3798461566 learned_addr learned my addr 172.21.15.96:0/3798461566 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:29.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcd5aa640 1 -- 172.21.15.96:0/3798461566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40094d0 con 0x7f5fc809dc20 2024-09-13T17:42:29.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcd5aa640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097f0 tx=0x7f5fc401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fc40b6420 con 0x7f5fc809dc20 2024-09-13T17:42:29.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40b6580 con 0x7f5fc809dc20 2024-09-13T17:42:29.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5fc40b6860 con 0x7f5fc809dc20 2024-09-13T17:42:29.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc8138f80 con 0x7f5fc809dc20 2024-09-13T17:42:29.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc811d4b0 con 0x7f5fc809dc20 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc8134ab0 con 0x7f5fc809dc20 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5fc40b2040 con 0x7f5fc809dc20 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 0x7f5fac037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5fc40e1950 con 0x7f5fc809dc20 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fccda9640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 0x7f5fac037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:29.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.977+0000 7f5fccda9640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 0x7f5fac037910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001600 tx=0x7f5fb8007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:29.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:29.981+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5fc40bd9b0 con 0x7f5fc809dc20 2024-09-13T17:42:30.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fc8099ee0 con 0x7f5fc809dc20 2024-09-13T17:42:30.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fbdffb640 1 -- 172.21.15.96:0/3798461566 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fc40b0070 con 0x7f5fc809dc20 2024-09-13T17:42:30.141 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 msgr2=0x7f5fac037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 0x7f5fac037910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8001600 tx=0x7f5fb8007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 msgr2=0x7f5fc80a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40097f0 tx=0x7f5fc401ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 shutdown_connections 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5fac035450 0x7f5fac037910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 --2- 172.21.15.96:0/3798461566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5fc809dc20 0x7f5fc80a3960 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 >> 172.21.15.96:0/3798461566 conn(0x7f5fc8098a60 msgr2=0x7f5fc8133e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 shutdown_connections 2024-09-13T17:42:30.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.141+0000 7f5fcf80e640 1 -- 172.21.15.96:0/3798461566 wait complete. 2024-09-13T17:42:30.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.281+0000 7f0add4a0640 1 Processor -- start 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.281+0000 7f0add4a0640 1 -- start start 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ad809e820 0x7f0ad813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad80a68b0 con 0x7f0ad8134e70 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad809ddc0 con 0x7f0ad809e820 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad67fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ad809e820 0x7f0ad813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0ad8134e70 0x7f0ad8135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43818/0 (socket says 172.21.15.96:43818) 2024-09-13T17:42:30.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad6ffd640 1 -- 172.21.15.96:0/1329677037 learned_addr learned my addr 172.21.15.96:0/1329677037 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:30.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad67fc640 1 -- 172.21.15.96:0/1329677037 >> v1:172.21.15.96:6789/0 conn(0x7f0ad8134e70 legacy=0x7f0ad8135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad67fc640 1 -- 172.21.15.96:0/1329677037 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8136970 con 0x7f0ad809e820 2024-09-13T17:42:30.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad67fc640 1 --2- 172.21.15.96:0/1329677037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad813ae50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009740 tx=0x7f0acc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeb719aab31eba5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad5ffb640 1 -- 172.21.15.96:0/1329677037 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0acc0ab530 con 0x7f0ad809e820 2024-09-13T17:42:30.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad5ffb640 1 -- 172.21.15.96:0/1329677037 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc0a7050 con 0x7f0ad809e820 2024-09-13T17:42:30.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0ad5ffb640 1 -- 172.21.15.96:0/1329677037 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0acc0ab7e0 con 0x7f0ad809e820 2024-09-13T17:42:30.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- 172.21.15.96:0/1329677037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 msgr2=0x7f0ad813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 --2- 172.21.15.96:0/1329677037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad813ae50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009740 tx=0x7f0acc01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- 172.21.15.96:0/1329677037 shutdown_connections 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 --2- 172.21.15.96:0/1329677037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad813ae50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- 172.21.15.96:0/1329677037 >> 172.21.15.96:0/1329677037 conn(0x7f0ad8098a60 msgr2=0x7f0ad809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- 172.21.15.96:0/1329677037 shutdown_connections 2024-09-13T17:42:30.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.285+0000 7f0add4a0640 1 -- 172.21.15.96:0/1329677037 wait complete. 2024-09-13T17:42:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 Processor -- start 2024-09-13T17:42:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 -- start start 2024-09-13T17:42:30.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad813a650 con 0x7f0ad809e820 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54868/0 (socket says 172.21.15.96:54868) 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad6ffd640 1 -- 172.21.15.96:0/3058210394 learned_addr learned my addr 172.21.15.96:0/3058210394 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad6ffd640 1 -- 172.21.15.96:0/3058210394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0acc009420 con 0x7f0ad809e820 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad6ffd640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c480 tx=0x7f0ac800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ac8005c20 con 0x7f0ad809e820 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8005d80 con 0x7f0ad809e820 2024-09-13T17:42:30.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ac80071a0 con 0x7f0ad809e820 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad811c930 con 0x7f0ad809e820 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad8071610 con 0x7f0ad809e820 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0ac8007300 con 0x7f0ad809e820 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad811cb30 con 0x7f0ad809e820 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 0x7f0ac4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f0ac8046b00 con 0x7f0ad809e820 2024-09-13T17:42:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad67fc640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 0x7f0ac4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.289+0000 7f0ad67fc640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 0x7f0ac4037910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0ad000 tx=0x7f0acc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.293+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0ac80075b0 con 0x7f0ad809e820 2024-09-13T17:42:30.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ad806a1b0 con 0x7f0ad809e820 2024-09-13T17:42:30.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0abf7fe640 1 -- 172.21.15.96:0/3058210394 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0ac8017030 con 0x7f0ad809e820 2024-09-13T17:42:30.453 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 msgr2=0x7f0ac4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 0x7f0ac4037910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0ad000 tx=0x7f0acc008040 comp rx=0 tx=0).stop 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 msgr2=0x7f0ad812ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c480 tx=0x7f0ac800c940 comp rx=0 tx=0).stop 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 shutdown_connections 2024-09-13T17:42:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0ac4035450 0x7f0ac4037910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 --2- 172.21.15.96:0/3058210394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ad809e820 0x7f0ad812ef70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 >> 172.21.15.96:0/3058210394 conn(0x7f0ad8098a60 msgr2=0x7f0ad8134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:30.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 shutdown_connections 2024-09-13T17:42:30.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.453+0000 7f0add4a0640 1 -- 172.21.15.96:0/3058210394 wait complete. 2024-09-13T17:42:30.473 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_fstop.TestFSTop.test_dump' 2024-09-13T17:42:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f772d640 1 Processor -- start 2024-09-13T17:42:30.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f772d640 1 -- start start 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f772d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55f013af50 0x7f55f013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f772d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f0138050 con 0x7f55f009d690 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f772d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f009c410 con 0x7f55f013af50 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f4cc8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55f013af50 0x7f55f013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f54c9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f55f009d690 0x7f55f0138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43820/0 (socket says 172.21.15.96:43820) 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f54c9640 1 -- 172.21.15.96:0/2555141180 learned_addr learned my addr 172.21.15.96:0/2555141180 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f4cc8640 1 -- 172.21.15.96:0/2555141180 >> v1:172.21.15.96:6789/0 conn(0x7f55f009d690 legacy=0x7f55f0138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f4cc8640 1 -- 172.21.15.96:0/2555141180 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f009cba0 con 0x7f55f013af50 2024-09-13T17:42:30.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55f4cc8640 1 --2- 172.21.15.96:0/2555141180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f013af50 0x7f55f013d340 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009740 tx=0x7f55e001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d184083e20658352 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55e7fff640 1 -- 172.21.15.96:0/2555141180 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55e00a8840 con 0x7f55f013af50 2024-09-13T17:42:30.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55e7fff640 1 -- 172.21.15.96:0/2555141180 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00a89a0 con 0x7f55f013af50 2024-09-13T17:42:30.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.593+0000 7f55e7fff640 1 -- 172.21.15.96:0/2555141180 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55e00a8c60 con 0x7f55f013af50 2024-09-13T17:42:30.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/2555141180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f013af50 msgr2=0x7f55f013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:30.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 --2- 172.21.15.96:0/2555141180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f013af50 0x7f55f013d340 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f55e0009740 tx=0x7f55e001ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:30.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/2555141180 shutdown_connections 2024-09-13T17:42:30.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 --2- 172.21.15.96:0/2555141180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f013af50 0x7f55f013d340 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:30.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/2555141180 >> 172.21.15.96:0/2555141180 conn(0x7f55f006bc60 msgr2=0x7f55f006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:30.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/2555141180 shutdown_connections 2024-09-13T17:42:30.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/2555141180 wait complete. 2024-09-13T17:42:30.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 Processor -- start 2024-09-13T17:42:30.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- start start 2024-09-13T17:42:30.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00a2d40 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f54c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f54c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54872/0 (socket says 172.21.15.96:54872) 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f54c9640 1 -- 172.21.15.96:0/727992307 learned_addr learned my addr 172.21.15.96:0/727992307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f54c9640 1 -- 172.21.15.96:0/727992307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e0009420 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f54c9640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c380 tx=0x7f55ec00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55ec005c20 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec005d80 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f01372e0 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55ec007200 con 0x7f55f009d690 2024-09-13T17:42:30.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f00a5100 con 0x7f55f009d690 2024-09-13T17:42:30.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f00a4f10 con 0x7f55f009d690 2024-09-13T17:42:30.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.597+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f55ec007360 con 0x7f55f009d690 2024-09-13T17:42:30.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.601+0000 7f55e5ffb640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 0x7f55d4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:30.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.601+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55ec0469d0 con 0x7f55f009d690 2024-09-13T17:42:30.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.601+0000 7f55f4cc8640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 0x7f55d4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:30.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.601+0000 7f55f4cc8640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 0x7f55d4037910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55e00b2000 tx=0x7f55e00aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:30.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.605+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f55ec007610 con 0x7f55f009d690 2024-09-13T17:42:30.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:30.809+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_dump"]} v 0) v1 -- 0x7f55f0138050 con 0x7f55f009d690 2024-09-13T17:42:31.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55e5ffb640 1 -- 172.21.15.96:0/727992307 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_dump"]}]=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7f55f0138050 con 0x7f55f009d690 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 msgr2=0x7f55d4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 0x7f55d4037910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f55e00b2000 tx=0x7f55e00aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 msgr2=0x7f55f01c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c380 tx=0x7f55ec00c840 comp rx=0 tx=0).stop 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 shutdown_connections 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.005+0000 7f55f772d640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f55d4035450 0x7f55d4037910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.009+0000 7f55f772d640 1 --2- 172.21.15.96:0/727992307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55f009d690 0x7f55f01c9550 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.009+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 >> 172.21.15.96:0/727992307 conn(0x7f55f006bc60 msgr2=0x7f55f0131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.009+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 shutdown_connections 2024-09-13T17:42:31.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.009+0000 7f55f772d640 1 -- 172.21.15.96:0/727992307 wait complete. 2024-09-13T17:42:31.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:42:31.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bf15a6640 1 Processor -- start 2024-09-13T17:42:31.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bf15a6640 1 -- start start 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bf15a6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5bec09e820 0x7f5bec13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bf15a6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec0a68b0 con 0x7f5bec134e70 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bf15a6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec09ddc0 con 0x7f5bec09e820 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5beaffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5bec134e70 0x7f5bec135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43832/0 (socket says 172.21.15.96:43832) 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5beaffd640 1 -- 172.21.15.96:0/3297010114 learned_addr learned my addr 172.21.15.96:0/3297010114 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:31.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.149+0000 7f5bea7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5bec09e820 0x7f5bec13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bea7fc640 1 -- 172.21.15.96:0/3297010114 >> v1:172.21.15.96:6789/0 conn(0x7f5bec134e70 legacy=0x7f5bec135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5be9ffb640 1 -- 172.21.15.96:0/3297010114 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1368150881 0 0) 0x7f5bec0a68b0 con 0x7f5bec134e70 2024-09-13T17:42:31.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bea7fc640 1 -- 172.21.15.96:0/3297010114 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec136970 con 0x7f5bec09e820 2024-09-13T17:42:31.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bea7fc640 1 --2- 172.21.15.96:0/3297010114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec13ae50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009d90 tx=0x7f5be001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d29a9889fcd30b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5be9ffb640 1 -- 172.21.15.96:0/3297010114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be00a57b0 con 0x7f5bec09e820 2024-09-13T17:42:31.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5be9ffb640 1 -- 172.21.15.96:0/3297010114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be00a5910 con 0x7f5bec09e820 2024-09-13T17:42:31.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5be9ffb640 1 -- 172.21.15.96:0/3297010114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be00a5bd0 con 0x7f5bec09e820 2024-09-13T17:42:31.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 -- 172.21.15.96:0/3297010114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 msgr2=0x7f5bec13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/3297010114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec13ae50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009d90 tx=0x7f5be001c970 comp rx=0 tx=0).stop 2024-09-13T17:42:31.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 -- 172.21.15.96:0/3297010114 shutdown_connections 2024-09-13T17:42:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/3297010114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec13ae50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 -- 172.21.15.96:0/3297010114 >> 172.21.15.96:0/3297010114 conn(0x7f5bec098a60 msgr2=0x7f5bec09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 -- 172.21.15.96:0/3297010114 shutdown_connections 2024-09-13T17:42:31.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.153+0000 7f5bf15a6640 1 -- 172.21.15.96:0/3297010114 wait complete. 2024-09-13T17:42:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 Processor -- start 2024-09-13T17:42:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 -- start start 2024-09-13T17:42:31.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec0a5aa0 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5beaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54886/0 (socket says 172.21.15.96:54886) 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5beaffd640 1 -- 172.21.15.96:0/338432026 learned_addr learned my addr 172.21.15.96:0/338432026 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5beaffd640 1 -- 172.21.15.96:0/338432026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5be0009a70 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5beaffd640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c420 tx=0x7f5be400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be40074d0 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be4007630 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec071270 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5be4004960 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec13a420 con 0x7f5bec09e820 2024-09-13T17:42:31.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec0715c0 con 0x7f5bec09e820 2024-09-13T17:42:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f5be4004ac0 con 0x7f5bec09e820 2024-09-13T17:42:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 0x7f5bd8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5be4046c30 con 0x7f5bec09e820 2024-09-13T17:42:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bea7fc640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 0x7f5bd8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.157+0000 7f5bea7fc640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 0x7f5bd8037910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5be00af000 tx=0x7f5be00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.161+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5be404d080 con 0x7f5bec09e820 2024-09-13T17:42:31.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.309+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5bec0a3c00 con 0x7f5bec09e820 2024-09-13T17:42:31.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bd37fe640 1 -- 172.21.15.96:0/338432026 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v5)=0 v5) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5be4018030 con 0x7f5bec09e820 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 msgr2=0x7f5bd8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 0x7f5bd8037910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5be00af000 tx=0x7f5be00a7070 comp rx=0 tx=0).stop 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 msgr2=0x7f5bec06f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5be400c420 tx=0x7f5be400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 shutdown_connections 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5bd8035450 0x7f5bd8037910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 --2- 172.21.15.96:0/338432026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bec09e820 0x7f5bec06f700 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 >> 172.21.15.96:0/338432026 conn(0x7f5bec098a60 msgr2=0x7f5bec133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 shutdown_connections 2024-09-13T17:42:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.313+0000 7f5bf15a6640 1 -- 172.21.15.96:0/338432026 wait complete. 2024-09-13T17:42:31.330 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:42:31.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:42:31.405 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:42:31.406 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:31.406 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:31.411 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:31.411 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:31.457 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:31.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.617+0000 7f8b79ab1640 1 Processor -- start 2024-09-13T17:42:31.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.617+0000 7f8b79ab1640 1 -- start start 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.617+0000 7f8b79ab1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8b7409dc20 0x7f8b741351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.617+0000 7f8b79ab1640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7406e130 con 0x7f8b7406f6d0 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.617+0000 7f8b79ab1640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740a6750 con 0x7f8b7409dc20 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b737fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8b7406f6d0 0x7f8b74134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43842/0 (socket says 172.21.15.96:43842) 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b737fe640 1 -- 172.21.15.96:0/141326934 learned_addr learned my addr 172.21.15.96:0/141326934 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b72ffd640 1 --2- 172.21.15.96:0/141326934 >> v2:172.21.15.96:3300/0 conn(0x7f8b7409dc20 0x7f8b741351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252765421 0 0) 0x7f8b7406e130 con 0x7f8b7406f6d0 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b5c003530 con 0x7f8b7406f6d0 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b72ffd640 1 -- 172.21.15.96:0/141326934 >> v1:172.21.15.96:6789/0 conn(0x7f8b7406f6d0 legacy=0x7f8b74134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373255472 0 0) 0x7f8b5c003530 con 0x7f8b7406f6d0 2024-09-13T17:42:31.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b72ffd640 1 -- 172.21.15.96:0/141326934 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740712e0 con 0x7f8b7409dc20 2024-09-13T17:42:31.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b72ffd640 1 --2- 172.21.15.96:0/141326934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7409dc20 0x7f8b741351c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009740 tx=0x7f8b6801c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cfeae855503c40 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b640015b0 con 0x7f8b7409dc20 2024-09-13T17:42:31.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b680a7050 con 0x7f8b7409dc20 2024-09-13T17:42:31.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b727fc640 1 -- 172.21.15.96:0/141326934 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b680ab400 con 0x7f8b7409dc20 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 -- 172.21.15.96:0/141326934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7409dc20 msgr2=0x7f8b741351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/141326934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7409dc20 0x7f8b741351c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009740 tx=0x7f8b6801c2b0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 -- 172.21.15.96:0/141326934 shutdown_connections 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/141326934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7409dc20 0x7f8b741351c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 -- 172.21.15.96:0/141326934 >> 172.21.15.96:0/141326934 conn(0x7f8b74098a60 msgr2=0x7f8b7409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 -- 172.21.15.96:0/141326934 shutdown_connections 2024-09-13T17:42:31.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.621+0000 7f8b79ab1640 1 -- 172.21.15.96:0/141326934 wait complete. 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 Processor -- start 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 -- start start 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b74132e70 con 0x7f8b7406f6d0 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54900/0 (socket says 172.21.15.96:54900) 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b737fe640 1 -- 172.21.15.96:0/3697708255 learned_addr learned my addr 172.21.15.96:0/3697708255 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b737fe640 1 -- 172.21.15.96:0/3697708255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68009420 con 0x7f8b7406f6d0 2024-09-13T17:42:31.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b737fe640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c6b0 tx=0x7f8b6400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b640058e0 con 0x7f8b7406f6d0 2024-09-13T17:42:31.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b64005a40 con 0x7f8b7406f6d0 2024-09-13T17:42:31.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7406f360 con 0x7f8b7406f6d0 2024-09-13T17:42:31.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8b64005ce0 con 0x7f8b7406f6d0 2024-09-13T17:42:31.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7409fe60 con 0x7f8b7406f6d0 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8b64007070 con 0x7f8b7406f6d0 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b740a06f0 con 0x7f8b7406f6d0 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 0x7f8b60037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8b64046d40 con 0x7f8b7406f6d0 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b72ffd640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 0x7f8b60037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.625+0000 7f8b72ffd640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 0x7f8b60037910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ad000 tx=0x7f8b680a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.629+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8b64010770 con 0x7f8b7406f6d0 2024-09-13T17:42:31.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.777+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b74134ab0 con 0x7f8b7406f6d0 2024-09-13T17:42:31.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.777+0000 7f8b5bfff640 1 -- 172.21.15.96:0/3697708255 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f8b64017070 con 0x7f8b7406f6d0 2024-09-13T17:42:31.780 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:31.780 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 msgr2=0x7f8b60037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 0x7f8b60037910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8b680ad000 tx=0x7f8b680a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 msgr2=0x7f8b7406e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8b6400c6b0 tx=0x7f8b6400cb70 comp rx=0 tx=0).stop 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 shutdown_connections 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8b60035450 0x7f8b60037910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 --2- 172.21.15.96:0/3697708255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b7406f6d0 0x7f8b7406e5c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 >> 172.21.15.96:0/3697708255 conn(0x7f8b74098a60 msgr2=0x7f8b74098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 shutdown_connections 2024-09-13T17:42:31.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.781+0000 7f8b79ab1640 1 -- 172.21.15.96:0/3697708255 wait complete. 2024-09-13T17:42:31.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:31.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.909+0000 7f36303bc640 1 Processor -- start 2024-09-13T17:42:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.909+0000 7f36303bc640 1 -- start start 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f362813af50 0x7f362813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3628138050 con 0x7f362809d690 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362809c410 con 0x7f362813af50 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d957640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f362813af50 0x7f362813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362e158640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f362809d690 0x7f3628138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43852/0 (socket says 172.21.15.96:43852) 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362e158640 1 -- 172.21.15.96:0/1183815823 learned_addr learned my addr 172.21.15.96:0/1183815823 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d957640 1 -- 172.21.15.96:0/1183815823 >> v1:172.21.15.96:6789/0 conn(0x7f362809d690 legacy=0x7f3628138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d957640 1 -- 172.21.15.96:0/1183815823 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362809cba0 con 0x7f362813af50 2024-09-13T17:42:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d156640 1 -- 172.21.15.96:0/1183815823 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 467826463 0 0) 0x7f3624001120 con 0x7f362809d690 2024-09-13T17:42:31.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d957640 1 --2- 172.21.15.96:0/1183815823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362813af50 0x7f362813d340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3618009740 tx=0x7f361801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130fc942968d89ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d156640 1 -- 172.21.15.96:0/1183815823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36180a8840 con 0x7f362813af50 2024-09-13T17:42:31.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d156640 1 -- 172.21.15.96:0/1183815823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36180a89a0 con 0x7f362813af50 2024-09-13T17:42:31.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f362d156640 1 -- 172.21.15.96:0/1183815823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36180a8c60 con 0x7f362813af50 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- 172.21.15.96:0/1183815823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362813af50 msgr2=0x7f362813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 --2- 172.21.15.96:0/1183815823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362813af50 0x7f362813d340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3618009740 tx=0x7f361801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- 172.21.15.96:0/1183815823 shutdown_connections 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 --2- 172.21.15.96:0/1183815823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362813af50 0x7f362813d340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- 172.21.15.96:0/1183815823 >> 172.21.15.96:0/1183815823 conn(0x7f362806bc60 msgr2=0x7f362806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- 172.21.15.96:0/1183815823 shutdown_connections 2024-09-13T17:42:31.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.913+0000 7f36303bc640 1 -- 172.21.15.96:0/1183815823 wait complete. 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 Processor -- start 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 -- start start 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36280a5c90 con 0x7f362809d690 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362e158640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362e158640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54926/0 (socket says 172.21.15.96:54926) 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362e158640 1 -- 172.21.15.96:0/2294062660 learned_addr learned my addr 172.21.15.96:0/2294062660 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362e158640 1 -- 172.21.15.96:0/2294062660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618009420 con 0x7f362809d690 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362e158640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f362400c390 tx=0x7f362400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3624007410 con 0x7f362809d690 2024-09-13T17:42:31.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624007590 con 0x7f362809d690 2024-09-13T17:42:31.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36280a6f20 con 0x7f362809d690 2024-09-13T17:42:31.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36240047f0 con 0x7f362809d690 2024-09-13T17:42:31.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3628137400 con 0x7f362809d690 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362809bc00 con 0x7f362809d690 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f3624004950 con 0x7f362809d690 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 0x7f360c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f36240472b0 con 0x7f362809d690 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362d957640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 0x7f360c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:31.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.917+0000 7f362d957640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 0x7f360c037910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36180b2000 tx=0x7f36180aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:31.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:31.921+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f362809bc00 con 0x7f362809d690 2024-09-13T17:42:32.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.073+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3628138050 con 0x7f362809d690 2024-09-13T17:42:32.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f361effd640 1 -- 172.21.15.96:0/2294062660 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3628138050 con 0x7f362809d690 2024-09-13T17:42:32.076 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:32.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 msgr2=0x7f360c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 0x7f360c037910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f36180b2000 tx=0x7f36180aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:32.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 msgr2=0x7f36280a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f362400c390 tx=0x7f362400c850 comp rx=0 tx=0).stop 2024-09-13T17:42:32.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 shutdown_connections 2024-09-13T17:42:32.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f360c035450 0x7f360c037910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 --2- 172.21.15.96:0/2294062660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f362809d690 0x7f36280a69e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 >> 172.21.15.96:0/2294062660 conn(0x7f362806bc60 msgr2=0x7f362809f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 shutdown_connections 2024-09-13T17:42:32.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.077+0000 7f36303bc640 1 -- 172.21.15.96:0/2294062660 wait complete. 2024-09-13T17:42:32.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:32.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49f021e640 1 Processor -- start 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49f021e640 1 -- start start 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49f021e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49e809e820 0x7f49e813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49f021e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e80a68b0 con 0x7f49e8134e70 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49f021e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e809ddc0 con 0x7f49e809e820 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49edfba640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f49e8134e70 0x7f49e8135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43868/0 (socket says 172.21.15.96:43868) 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49edfba640 1 -- 172.21.15.96:0/3534250968 learned_addr learned my addr 172.21.15.96:0/3534250968 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49ed7b9640 1 --2- 172.21.15.96:0/3534250968 >> v2:172.21.15.96:3300/0 conn(0x7f49e809e820 0x7f49e813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49ecfb8640 1 -- 172.21.15.96:0/3534250968 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878404402 0 0) 0x7f49e80a68b0 con 0x7f49e8134e70 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49ecfb8640 1 -- 172.21.15.96:0/3534250968 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49d4003530 con 0x7f49e8134e70 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49ed7b9640 1 -- 172.21.15.96:0/3534250968 >> v1:172.21.15.96:6789/0 conn(0x7f49e8134e70 legacy=0x7f49e8135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.217+0000 7f49ed7b9640 1 -- 172.21.15.96:0/3534250968 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e8136970 con 0x7f49e809e820 2024-09-13T17:42:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49ed7b9640 1 --2- 172.21.15.96:0/3534250968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e813ae50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009740 tx=0x7f49e401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5c2de4e621987e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49ecfb8640 1 -- 172.21.15.96:0/3534250968 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e40ab530 con 0x7f49e809e820 2024-09-13T17:42:32.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49ecfb8640 1 -- 172.21.15.96:0/3534250968 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e40a7050 con 0x7f49e809e820 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49ecfb8640 1 -- 172.21.15.96:0/3534250968 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49e40ab7c0 con 0x7f49e809e820 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- 172.21.15.96:0/3534250968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 msgr2=0x7f49e813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 --2- 172.21.15.96:0/3534250968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e813ae50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009740 tx=0x7f49e401c420 comp rx=0 tx=0).stop 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- 172.21.15.96:0/3534250968 shutdown_connections 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 --2- 172.21.15.96:0/3534250968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e813ae50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- 172.21.15.96:0/3534250968 >> 172.21.15.96:0/3534250968 conn(0x7f49e8098a60 msgr2=0x7f49e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- 172.21.15.96:0/3534250968 shutdown_connections 2024-09-13T17:42:32.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- 172.21.15.96:0/3534250968 wait complete. 2024-09-13T17:42:32.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 Processor -- start 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.221+0000 7f49f021e640 1 -- start start 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49f021e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49f021e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80a10d0 con 0x7f49e809e820 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49edfba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49edfba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54948/0 (socket says 172.21.15.96:54948) 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49edfba640 1 -- 172.21.15.96:0/2186930958 learned_addr learned my addr 172.21.15.96:0/2186930958 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49edfba640 1 -- 172.21.15.96:0/2186930958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4009420 con 0x7f49e809e820 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49edfba640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3d0 tx=0x7f49d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d8005ce0 con 0x7f49e809e820 2024-09-13T17:42:32.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8005e40 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e80a2c80 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d8007220 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e8132fe0 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f49d8007440 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e80008e0 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 0x7f49cc0378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f49d8046d50 con 0x7f49e809e820 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49ed7b9640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 0x7f49cc0378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.225+0000 7f49ed7b9640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 0x7f49cc0378c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49e40ad000 tx=0x7f49e40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.229+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49d8019b40 con 0x7f49e809e820 2024-09-13T17:42:32.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49e80a29c0 con 0x7f49e809e820 2024-09-13T17:42:32.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49de7fc640 1 -- 172.21.15.96:0/2186930958 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49d8017070 con 0x7f49e809e820 2024-09-13T17:42:32.389 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:32.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 msgr2=0x7f49cc0378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 0x7f49cc0378c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49e40ad000 tx=0x7f49e40a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:32.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 msgr2=0x7f49e809f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f49d800c3d0 tx=0x7f49d800c890 comp rx=0 tx=0).stop 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 shutdown_connections 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f49cc035400 0x7f49cc0378c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 --2- 172.21.15.96:0/2186930958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e809e820 0x7f49e809f280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 >> 172.21.15.96:0/2186930958 conn(0x7f49e8098a60 msgr2=0x7f49e806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 shutdown_connections 2024-09-13T17:42:32.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.389+0000 7f49f021e640 1 -- 172.21.15.96:0/2186930958 wait complete. 2024-09-13T17:42:32.408 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:32.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.525+0000 7f625f0f6640 1 Processor -- start 2024-09-13T17:42:32.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.525+0000 7f625f0f6640 1 -- start start 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f625813af50 0x7f625813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6258138050 con 0x7f625809d690 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f625809c410 con 0x7f625813af50 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625ce92640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f625809d690 0x7f6258138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33624/0 (socket says 172.21.15.96:33624) 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625ce92640 1 -- 172.21.15.96:0/118973179 learned_addr learned my addr 172.21.15.96:0/118973179 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624ffff640 1 --2- 172.21.15.96:0/118973179 >> v2:172.21.15.96:3300/0 conn(0x7f625813af50 0x7f625813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624f7fe640 1 -- 172.21.15.96:0/118973179 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103344188 0 0) 0x7f6254001120 con 0x7f625809d690 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624f7fe640 1 -- 172.21.15.96:0/118973179 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6240003530 con 0x7f625809d690 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624ffff640 1 -- 172.21.15.96:0/118973179 >> v1:172.21.15.96:6789/0 conn(0x7f625809d690 legacy=0x7f6258138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624ffff640 1 -- 172.21.15.96:0/118973179 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625809cba0 con 0x7f625813af50 2024-09-13T17:42:32.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624ffff640 1 --2- 172.21.15.96:0/118973179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625813af50 0x7f625813d340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6248009740 tx=0x7f624801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974c2781343d16ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624f7fe640 1 -- 172.21.15.96:0/118973179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62480a8840 con 0x7f625813af50 2024-09-13T17:42:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624f7fe640 1 -- 172.21.15.96:0/118973179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62480a89a0 con 0x7f625813af50 2024-09-13T17:42:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f624f7fe640 1 -- 172.21.15.96:0/118973179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62480a8c60 con 0x7f625813af50 2024-09-13T17:42:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- 172.21.15.96:0/118973179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625813af50 msgr2=0x7f625813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 --2- 172.21.15.96:0/118973179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625813af50 0x7f625813d340 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6248009740 tx=0x7f624801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- 172.21.15.96:0/118973179 shutdown_connections 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 --2- 172.21.15.96:0/118973179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625813af50 0x7f625813d340 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- 172.21.15.96:0/118973179 >> 172.21.15.96:0/118973179 conn(0x7f625806bc60 msgr2=0x7f625806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- 172.21.15.96:0/118973179 shutdown_connections 2024-09-13T17:42:32.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.529+0000 7f625f0f6640 1 -- 172.21.15.96:0/118973179 wait complete. 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 Processor -- start 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 -- start start 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62580a43e0 con 0x7f625809d690 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625ce92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625ce92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48450/0 (socket says 172.21.15.96:48450) 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625ce92640 1 -- 172.21.15.96:0/3639574925 learned_addr learned my addr 172.21.15.96:0/3639574925 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:32.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625ce92640 1 -- 172.21.15.96:0/3639574925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009420 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625ce92640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f625400c3a0 tx=0x7f625400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6254005c20 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6254005d80 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62580a61b0 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6254007200 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62580a70c0 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f6254007360 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625809b220 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f62540469a0 con 0x7f625809d690 2024-09-13T17:42:32.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624ffff640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.533+0000 7f624ffff640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62480b2000 tx=0x7f62480aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.537+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6254007610 con 0x7f625809d690 2024-09-13T17:42:32.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.681+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6258138050 con 0x7f625809d690 2024-09-13T17:42:32.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.681+0000 7f624d7fa640 1 -- 172.21.15.96:0/3639574925 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6258138050 con 0x7f625809d690 2024-09-13T17:42:32.684 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:32.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 msgr2=0x7f6244037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f62480b2000 tx=0x7f62480aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 msgr2=0x7f62580a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f625400c3a0 tx=0x7f625400c860 comp rx=0 tx=0).stop 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 shutdown_connections 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 --2- 172.21.15.96:0/3639574925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f625809d690 0x7f62580a6b80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 >> 172.21.15.96:0/3639574925 conn(0x7f625806bc60 msgr2=0x7f62581c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 shutdown_connections 2024-09-13T17:42:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.685+0000 7f625f0f6640 1 -- 172.21.15.96:0/3639574925 wait complete. 2024-09-13T17:42:32.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:32.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fca1e6640 1 Processor -- start 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fca1e6640 1 -- start start 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fca1e6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6fc409d690 0x7f6fc4138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fca1e6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc4138050 con 0x7f6fc41387d0 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fca1e6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc409c410 con 0x7f6fc409d690 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fc37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6fc409d690 0x7f6fc4138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fc37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6fc409d690 0x7f6fc4138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48462/0 (socket says 172.21.15.96:48462) 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fc37fe640 1 -- 172.21.15.96:0/3688243460 learned_addr learned my addr 172.21.15.96:0/3688243460 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:32.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fc2ffd640 1 --1- 172.21.15.96:0/3688243460 >> v1:172.21.15.96:6789/0 conn(0x7f6fc41387d0 0x7f6fc4134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33638/0 (socket says 172.21.15.96:33638) 2024-09-13T17:42:32.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.809+0000 7f6fc37fe640 1 -- 172.21.15.96:0/3688243460 >> v1:172.21.15.96:6789/0 conn(0x7f6fc41387d0 legacy=0x7f6fc4134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fc37fe640 1 -- 172.21.15.96:0/3688243460 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc409cba0 con 0x7f6fc409d690 2024-09-13T17:42:32.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fc37fe640 1 --2- 172.21.15.96:0/3688243460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc4138290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097f0 tx=0x7f6fb401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b215b8e697ec02c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fc27fc640 1 -- 172.21.15.96:0/3688243460 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb40a8900 con 0x7f6fc409d690 2024-09-13T17:42:32.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fc27fc640 1 -- 172.21.15.96:0/3688243460 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40a8a60 con 0x7f6fc409d690 2024-09-13T17:42:32.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fc27fc640 1 -- 172.21.15.96:0/3688243460 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb40a8d20 con 0x7f6fc409d690 2024-09-13T17:42:32.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 -- 172.21.15.96:0/3688243460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 msgr2=0x7f6fc4138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/3688243460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc4138290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40097f0 tx=0x7f6fb401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 -- 172.21.15.96:0/3688243460 shutdown_connections 2024-09-13T17:42:32.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/3688243460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc4138290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 -- 172.21.15.96:0/3688243460 >> 172.21.15.96:0/3688243460 conn(0x7f6fc406bc60 msgr2=0x7f6fc406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 -- 172.21.15.96:0/3688243460 shutdown_connections 2024-09-13T17:42:32.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 -- 172.21.15.96:0/3688243460 wait complete. 2024-09-13T17:42:32.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.813+0000 7f6fca1e6640 1 Processor -- start 2024-09-13T17:42:32.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 -- start start 2024-09-13T17:42:32.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc40a5d50 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48470/0 (socket says 172.21.15.96:48470) 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc37fe640 1 -- 172.21.15.96:0/2895988293 learned_addr learned my addr 172.21.15.96:0/2895988293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc37fe640 1 -- 172.21.15.96:0/2895988293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40094d0 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc37fe640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fb401c900 tx=0x7f6fb401ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb40bf070 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40b09b0 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc409f400 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fb40b0c70 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc40a6060 con 0x7f6fc409d690 2024-09-13T17:42:32.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f6fb40b7070 con 0x7f6fc409d690 2024-09-13T17:42:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc409bc00 con 0x7f6fc409d690 2024-09-13T17:42:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 0x7f6fb0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f6fb40e4f90 con 0x7f6fc409d690 2024-09-13T17:42:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc2ffd640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 0x7f6fb0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:32.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.817+0000 7f6fc2ffd640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 0x7f6fb0037910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4067370 tx=0x7f6fb8007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:32.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.821+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6fc409bc00 con 0x7f6fc409d690 2024-09-13T17:42:32.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.985+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fc40a2bd0 con 0x7f6fc409d690 2024-09-13T17:42:32.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.985+0000 7f6fabfff640 1 -- 172.21.15.96:0/2895988293 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6fc40a2bd0 con 0x7f6fc409d690 2024-09-13T17:42:32.987 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 msgr2=0x7f6fb0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 0x7f6fb0037910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4067370 tx=0x7f6fb8007a10 comp rx=0 tx=0).stop 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 msgr2=0x7f6fc40a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6fb401c900 tx=0x7f6fb401ba70 comp rx=0 tx=0).stop 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 shutdown_connections 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6fb0035450 0x7f6fb0037910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 --2- 172.21.15.96:0/2895988293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fc409d690 0x7f6fc40a6aa0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 >> 172.21.15.96:0/2895988293 conn(0x7f6fc406bc60 msgr2=0x7f6fc409e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 shutdown_connections 2024-09-13T17:42:32.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:32.989+0000 7f6fca1e6640 1 -- 172.21.15.96:0/2895988293 wait complete. 2024-09-13T17:42:33.005 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:33.005 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:33.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f84df640 1 Processor -- start 2024-09-13T17:42:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f84df640 1 -- start start 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f84df640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb5f0134670 0x7fb5f0134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f84df640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f0071670 con 0x7fb5f006e5c0 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f84df640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f0071360 con 0x7fb5f0134670 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f5a7a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb5f006e5c0 0x7fb5f006ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33644/0 (socket says 172.21.15.96:33644) 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f627b640 1 --2- 172.21.15.96:0/1995166588 >> v2:172.21.15.96:3300/0 conn(0x7fb5f0134670 0x7fb5f0134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f5a7a640 1 -- 172.21.15.96:0/1995166588 learned_addr learned my addr 172.21.15.96:0/1995166588 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f627b640 1 -- 172.21.15.96:0/1995166588 >> v1:172.21.15.96:6789/0 conn(0x7fb5f006e5c0 legacy=0x7fb5f006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.169+0000 7fb5f627b640 1 -- 172.21.15.96:0/1995166588 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00a7200 con 0x7fb5f0134670 2024-09-13T17:42:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f627b640 1 --2- 172.21.15.96:0/1995166588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f0134670 0x7fb5f0134a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009740 tx=0x7fb5e001d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3303c08b2a9d80ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f5279640 1 -- 172.21.15.96:0/1995166588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5e00ac530 con 0x7fb5f0134670 2024-09-13T17:42:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f5279640 1 -- 172.21.15.96:0/1995166588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e00a8050 con 0x7fb5f0134670 2024-09-13T17:42:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f5279640 1 -- 172.21.15.96:0/1995166588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5e00ac7e0 con 0x7fb5f0134670 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- 172.21.15.96:0/1995166588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f0134670 msgr2=0x7fb5f0134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 --2- 172.21.15.96:0/1995166588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f0134670 0x7fb5f0134a50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009740 tx=0x7fb5e001d4a0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- 172.21.15.96:0/1995166588 shutdown_connections 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 --2- 172.21.15.96:0/1995166588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f0134670 0x7fb5f0134a50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- 172.21.15.96:0/1995166588 >> 172.21.15.96:0/1995166588 conn(0x7fb5f0098a60 msgr2=0x7fb5f009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- 172.21.15.96:0/1995166588 shutdown_connections 2024-09-13T17:42:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- 172.21.15.96:0/1995166588 wait complete. 2024-09-13T17:42:33.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 Processor -- start 2024-09-13T17:42:33.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.173+0000 7fb5f84df640 1 -- start start 2024-09-13T17:42:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f84df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f84df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f009f700 con 0x7fb5f006e5c0 2024-09-13T17:42:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f627b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f627b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48494/0 (socket says 172.21.15.96:48494) 2024-09-13T17:42:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f627b640 1 -- 172.21.15.96:0/2725731973 learned_addr learned my addr 172.21.15.96:0/2725731973 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:33.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f627b640 1 -- 172.21.15.96:0/2725731973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e0009420 con 0x7fb5f006e5c0 2024-09-13T17:42:33.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f627b640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001b0d0 tx=0x7fb5e001a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5e00b5530 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e00b5690 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb5e00b5930 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f009f890 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f00a1720 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f009d520 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb5e00b3040 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 0x7fb5d4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb5e00e2a60 con 0x7fb5f006e5c0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f5a7a640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 0x7fb5d4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.177+0000 7fb5f5a7a640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 0x7fb5d4037910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec001610 tx=0x7fb5ec007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.181+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb5e00b5a90 con 0x7fb5f006e5c0 2024-09-13T17:42:33.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.329+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5f006a1b0 con 0x7fb5f006e5c0 2024-09-13T17:42:33.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5e6ffd640 1 -- 172.21.15.96:0/2725731973 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fb5e00b1020 con 0x7fb5f006e5c0 2024-09-13T17:42:33.332 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:33.332 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:33.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 msgr2=0x7fb5d4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 0x7fb5d4037910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec001610 tx=0x7fb5ec007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 msgr2=0x7fb5f00a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001b0d0 tx=0x7fb5e001a240 comp rx=0 tx=0).stop 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 shutdown_connections 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb5d4035450 0x7fb5d4037910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 --2- 172.21.15.96:0/2725731973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5f006e5c0 0x7fb5f00a0250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 >> 172.21.15.96:0/2725731973 conn(0x7fb5f0098a60 msgr2=0x7fb5f0132f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 shutdown_connections 2024-09-13T17:42:33.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.333+0000 7fb5f84df640 1 -- 172.21.15.96:0/2725731973 wait complete. 2024-09-13T17:42:33.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:33.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:33.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1653cab640 1 Processor -- start 2024-09-13T17:42:33.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1653cab640 1 -- start start 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1653cab640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f164c09d690 0x7f164c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1653cab640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164c138050 con 0x7f164c1387d0 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1653cab640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164c09c410 con 0x7f164c09d690 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1651a47640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f164c09d690 0x7f164c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1651246640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f164c1387d0 0x7f164c134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33646/0 (socket says 172.21.15.96:33646) 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.468+0000 7f1651246640 1 -- 172.21.15.96:0/537106326 learned_addr learned my addr 172.21.15.96:0/537106326 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1651a47640 1 -- 172.21.15.96:0/537106326 >> v1:172.21.15.96:6789/0 conn(0x7f164c1387d0 legacy=0x7f164c134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1651a47640 1 -- 172.21.15.96:0/537106326 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c09cba0 con 0x7f164c09d690 2024-09-13T17:42:33.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1651a47640 1 --2- 172.21.15.96:0/537106326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c138290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1648009740 tx=0x7f164801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abea204c0036f780 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1650a45640 1 -- 172.21.15.96:0/537106326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16480a8840 con 0x7f164c09d690 2024-09-13T17:42:33.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1650a45640 1 -- 172.21.15.96:0/537106326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16480a89a0 con 0x7f164c09d690 2024-09-13T17:42:33.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1650a45640 1 -- 172.21.15.96:0/537106326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16480a8c60 con 0x7f164c09d690 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 -- 172.21.15.96:0/537106326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 msgr2=0x7f164c138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 --2- 172.21.15.96:0/537106326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c138290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1648009740 tx=0x7f164801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 -- 172.21.15.96:0/537106326 shutdown_connections 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 --2- 172.21.15.96:0/537106326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c138290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 -- 172.21.15.96:0/537106326 >> 172.21.15.96:0/537106326 conn(0x7f164c06bc60 msgr2=0x7f164c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 -- 172.21.15.96:0/537106326 shutdown_connections 2024-09-13T17:42:33.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.472+0000 7f1653cab640 1 -- 172.21.15.96:0/537106326 wait complete. 2024-09-13T17:42:33.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 Processor -- start 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 -- start start 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0a4d00 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651a47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651a47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48526/0 (socket says 172.21.15.96:48526) 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651a47640 1 -- 172.21.15.96:0/4196542392 learned_addr learned my addr 172.21.15.96:0/4196542392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651a47640 1 -- 172.21.15.96:0/4196542392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1648009420 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651a47640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f16480b3040 tx=0x7f164801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16480b0800 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16480b0960 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16480b0c00 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c0a4f30 con 0x7f164c09d690 2024-09-13T17:42:33.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c0a6f00 con 0x7f164c09d690 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f16480b7040 con 0x7f164c09d690 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c0a4420 con 0x7f164c09d690 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 0x7f1630037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f16480e5320 con 0x7f164c09d690 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651246640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 0x7f1630037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.476+0000 7f1651246640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 0x7f1630037910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f164c067370 tx=0x7f163c007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.480+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f16480af3a0 con 0x7f164c09d690 2024-09-13T17:42:33.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.628+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f164c09da70 con 0x7f164c09d690 2024-09-13T17:42:33.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.628+0000 7f16427fc640 1 -- 172.21.15.96:0/4196542392 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f16480b5070 con 0x7f164c09d690 2024-09-13T17:42:33.630 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 msgr2=0x7f1630037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 0x7f1630037910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f164c067370 tx=0x7f163c007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 msgr2=0x7f164c0a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f16480b3040 tx=0x7f164801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 shutdown_connections 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1630035450 0x7f1630037910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 --2- 172.21.15.96:0/4196542392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f164c09d690 0x7f164c0a69c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 >> 172.21.15.96:0/4196542392 conn(0x7f164c06bc60 msgr2=0x7f164c0993a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 shutdown_connections 2024-09-13T17:42:33.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.632+0000 7f1653cab640 1 -- 172.21.15.96:0/4196542392 wait complete. 2024-09-13T17:42:33.647 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb27e553640 1 Processor -- start 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb27e553640 1 -- start start 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb27e553640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb27809e820 0x7fb27813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb27e553640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2780a68b0 con 0x7fb278134e70 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb27e553640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27809ddc0 con 0x7fb27809e820 2024-09-13T17:42:33.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb2777fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb27809e820 0x7fb27813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb277fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb278134e70 0x7fb278135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33650/0 (socket says 172.21.15.96:33650) 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb277fff640 1 -- 172.21.15.96:0/731359194 learned_addr learned my addr 172.21.15.96:0/731359194 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb276ffd640 1 -- 172.21.15.96:0/731359194 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445300085 0 0) 0x7fb2780a68b0 con 0x7fb278134e70 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb276ffd640 1 -- 172.21.15.96:0/731359194 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb260003530 con 0x7fb278134e70 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb2777fe640 1 -- 172.21.15.96:0/731359194 >> v1:172.21.15.96:6789/0 conn(0x7fb278134e70 legacy=0x7fb278135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.764+0000 7fb2777fe640 1 -- 172.21.15.96:0/731359194 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278136970 con 0x7fb27809e820 2024-09-13T17:42:33.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb2777fe640 1 --2- 172.21.15.96:0/731359194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27813ae50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb268009740 tx=0x7fb26801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436b75b1fa4b6735 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb276ffd640 1 -- 172.21.15.96:0/731359194 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2680ab530 con 0x7fb27809e820 2024-09-13T17:42:33.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb276ffd640 1 -- 172.21.15.96:0/731359194 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2680a7050 con 0x7fb27809e820 2024-09-13T17:42:33.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb276ffd640 1 -- 172.21.15.96:0/731359194 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2680ab7c0 con 0x7fb27809e820 2024-09-13T17:42:33.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/731359194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 msgr2=0x7fb27813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 --2- 172.21.15.96:0/731359194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27813ae50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb268009740 tx=0x7fb26801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:33.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/731359194 shutdown_connections 2024-09-13T17:42:33.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 --2- 172.21.15.96:0/731359194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27813ae50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/731359194 >> 172.21.15.96:0/731359194 conn(0x7fb278098a60 msgr2=0x7fb27809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/731359194 shutdown_connections 2024-09-13T17:42:33.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/731359194 wait complete. 2024-09-13T17:42:33.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 Processor -- start 2024-09-13T17:42:33.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- start start 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780a10d0 con 0x7fb27809e820 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb277fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb277fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48552/0 (socket says 172.21.15.96:48552) 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb277fff640 1 -- 172.21.15.96:0/1861044962 learned_addr learned my addr 172.21.15.96:0/1861044962 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb277fff640 1 -- 172.21.15.96:0/1861044962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb268009420 con 0x7fb27809e820 2024-09-13T17:42:33.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb277fff640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c3d0 tx=0x7fb26c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb26c005ce0 con 0x7fb27809e820 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26c005e40 con 0x7fb27809e820 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb26c007220 con 0x7fb27809e820 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2780a2c80 con 0x7fb27809e820 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.768+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb278132fe0 con 0x7fb27809e820 2024-09-13T17:42:33.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb26c007440 con 0x7fb27809e820 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27813ab00 con 0x7fb27809e820 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb274ff9640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 0x7fb2640378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb26c046d50 con 0x7fb27809e820 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb2777fe640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 0x7fb2640378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.772+0000 7fb2777fe640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 0x7fb2640378c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2680ad000 tx=0x7fb2680a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:33.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.776+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb27813ab00 con 0x7fb27809e820 2024-09-13T17:42:33.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.920+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb27813ab00 con 0x7fb27809e820 2024-09-13T17:42:33.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.920+0000 7fb274ff9640 1 -- 172.21.15.96:0/1861044962 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb27813ab00 con 0x7fb27809e820 2024-09-13T17:42:33.923 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 msgr2=0x7fb2640378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 0x7fb2640378c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb2680ad000 tx=0x7fb2680a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 msgr2=0x7fb27809f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c3d0 tx=0x7fb26c00c890 comp rx=0 tx=0).stop 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 shutdown_connections 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb264035400 0x7fb2640378c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 --2- 172.21.15.96:0/1861044962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb27809e820 0x7fb27809f280 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 >> 172.21.15.96:0/1861044962 conn(0x7fb278098a60 msgr2=0x7fb27806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 shutdown_connections 2024-09-13T17:42:33.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:33.924+0000 7fb27e553640 1 -- 172.21.15.96:0/1861044962 wait complete. 2024-09-13T17:42:33.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:34.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.052+0000 7f792f2fb640 1 Processor -- start 2024-09-13T17:42:34.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.052+0000 7f792f2fb640 1 -- start start 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f792809d690 0x7f7928138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7928138050 con 0x7f79281387d0 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792809c410 con 0x7f792809d690 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792d097640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f792809d690 0x7f7928138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792c896640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f79281387d0 0x7f7928134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33664/0 (socket says 172.21.15.96:33664) 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792c896640 1 -- 172.21.15.96:0/351889534 learned_addr learned my addr 172.21.15.96:0/351889534 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f791ffff640 1 -- 172.21.15.96:0/351889534 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551684508 0 0) 0x7f7918001120 con 0x7f79281387d0 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f791ffff640 1 -- 172.21.15.96:0/351889534 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7914003530 con 0x7f79281387d0 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792d097640 1 -- 172.21.15.96:0/351889534 >> v1:172.21.15.96:6789/0 conn(0x7f79281387d0 legacy=0x7f7928134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792d097640 1 -- 172.21.15.96:0/351889534 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792809cba0 con 0x7f792809d690 2024-09-13T17:42:34.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792d097640 1 --2- 172.21.15.96:0/351889534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f7928138290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7924009740 tx=0x7f792401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6724da33afe834 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f791ffff640 1 -- 172.21.15.96:0/351889534 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79240a8840 con 0x7f792809d690 2024-09-13T17:42:34.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f791ffff640 1 -- 172.21.15.96:0/351889534 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240a89a0 con 0x7f792809d690 2024-09-13T17:42:34.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f791ffff640 1 -- 172.21.15.96:0/351889534 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79240a8c40 con 0x7f792809d690 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- 172.21.15.96:0/351889534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 msgr2=0x7f7928138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 --2- 172.21.15.96:0/351889534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f7928138290 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7924009740 tx=0x7f792401ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- 172.21.15.96:0/351889534 shutdown_connections 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 --2- 172.21.15.96:0/351889534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f7928138290 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- 172.21.15.96:0/351889534 >> 172.21.15.96:0/351889534 conn(0x7f792806bc60 msgr2=0x7f792806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- 172.21.15.96:0/351889534 shutdown_connections 2024-09-13T17:42:34.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.056+0000 7f792f2fb640 1 -- 172.21.15.96:0/351889534 wait complete. 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 Processor -- start 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 -- start start 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79280a6060 con 0x7f792809d690 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792d097640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792d097640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48566/0 (socket says 172.21.15.96:48566) 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792d097640 1 -- 172.21.15.96:0/3013932218 learned_addr learned my addr 172.21.15.96:0/3013932218 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792d097640 1 -- 172.21.15.96:0/3013932218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924009420 con 0x7f792809d690 2024-09-13T17:42:34.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792d097640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f792401b820 tx=0x7f792401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7924004250 con 0x7f792809d690 2024-09-13T17:42:34.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240043b0 con 0x7f792809d690 2024-09-13T17:42:34.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7924003620 con 0x7f792809d690 2024-09-13T17:42:34.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79280a4490 con 0x7f792809d690 2024-09-13T17:42:34.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79280a41b0 con 0x7f792809d690 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f79240b7040 con 0x7f792809d690 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 0x7f790c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f79240e5230 con 0x7f792809d690 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792809bc00 con 0x7f792809d690 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792c896640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 0x7f790c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.060+0000 7f792c896640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 0x7f790c037910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7918005890 tx=0x7f7918001190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.064+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f792809bc00 con 0x7f792809d690 2024-09-13T17:42:34.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.208+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f792809da70 con 0x7f792809d690 2024-09-13T17:42:34.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.208+0000 7f791dffb640 1 -- 172.21.15.96:0/3013932218 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79240b5020 con 0x7f792809d690 2024-09-13T17:42:34.211 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 msgr2=0x7f790c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 0x7f790c037910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7918005890 tx=0x7f7918001190 comp rx=0 tx=0).stop 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 msgr2=0x7f79280a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f792401b820 tx=0x7f792401b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 shutdown_connections 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f790c035450 0x7f790c037910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 --2- 172.21.15.96:0/3013932218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f792809d690 0x7f79280a6aa0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 >> 172.21.15.96:0/3013932218 conn(0x7f792806bc60 msgr2=0x7f79281bff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:34.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 shutdown_connections 2024-09-13T17:42:34.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.212+0000 7f792f2fb640 1 -- 172.21.15.96:0/3013932218 wait complete. 2024-09-13T17:42:34.228 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:34.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b1dee8640 1 Processor -- start 2024-09-13T17:42:34.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b1dee8640 1 -- start start 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b1dee8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1b1809e900 0x7f1b1813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b1dee8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b1806ef60 con 0x7f1b18135f10 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b1dee8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1809ed10 con 0x7f1b1809e900 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b0ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1b1809e900 0x7f1b1813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b177fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1b18135f10 0x7f1b181362f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33674/0 (socket says 172.21.15.96:33674) 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b177fe640 1 -- 172.21.15.96:0/671626508 learned_addr learned my addr 172.21.15.96:0/671626508 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b0ffff640 1 -- 172.21.15.96:0/671626508 >> v1:172.21.15.96:6789/0 conn(0x7f1b18135f10 legacy=0x7f1b181362f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b16ffd640 1 -- 172.21.15.96:0/671626508 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940524620 0 0) 0x7f1b1806ef60 con 0x7f1b18135f10 2024-09-13T17:42:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.344+0000 7f1b0ffff640 1 -- 172.21.15.96:0/671626508 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18136a00 con 0x7f1b1809e900 2024-09-13T17:42:34.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b0ffff640 1 --2- 172.21.15.96:0/671626508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1813ade0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009740 tx=0x7f1b0001cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b910c17d1534a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b16ffd640 1 -- 172.21.15.96:0/671626508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b000ab530 con 0x7f1b1809e900 2024-09-13T17:42:34.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b16ffd640 1 -- 172.21.15.96:0/671626508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b000a7050 con 0x7f1b1809e900 2024-09-13T17:42:34.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b16ffd640 1 -- 172.21.15.96:0/671626508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b000ab7e0 con 0x7f1b1809e900 2024-09-13T17:42:34.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- 172.21.15.96:0/671626508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 msgr2=0x7f1b1813ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/671626508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1813ade0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009740 tx=0x7f1b0001cdf0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- 172.21.15.96:0/671626508 shutdown_connections 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/671626508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1813ade0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- 172.21.15.96:0/671626508 >> 172.21.15.96:0/671626508 conn(0x7f1b18066820 msgr2=0x7f1b1809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- 172.21.15.96:0/671626508 shutdown_connections 2024-09-13T17:42:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- 172.21.15.96:0/671626508 wait complete. 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 Processor -- start 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.348+0000 7f1b1dee8640 1 -- start start 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b1dee8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b1dee8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180709d0 con 0x7f1b1809e900 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b177fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48584/0 (socket says 172.21.15.96:48584) 2024-09-13T17:42:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b177fe640 1 -- 172.21.15.96:0/2506546457 learned_addr learned my addr 172.21.15.96:0/2506546457 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:34.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b177fe640 1 -- 172.21.15.96:0/2506546457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00009420 con 0x7f1b1809e900 2024-09-13T17:42:34.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b177fe640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c450 tx=0x7f1b0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b08005c20 con 0x7f1b1809e900 2024-09-13T17:42:34.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08005d80 con 0x7f1b1809e900 2024-09-13T17:42:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b080071a0 con 0x7f1b1809e900 2024-09-13T17:42:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b18071180 con 0x7f1b1809e900 2024-09-13T17:42:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b18133f10 con 0x7f1b1809e900 2024-09-13T17:42:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f1b08007300 con 0x7f1b1809e900 2024-09-13T17:42:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b18130a50 con 0x7f1b1809e900 2024-09-13T17:42:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 0x7f1af8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f1b08047250 con 0x7f1b1809e900 2024-09-13T17:42:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b0ffff640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 0x7f1af8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.352+0000 7f1b0ffff640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 0x7f1af8037910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ad000 tx=0x7f1b00008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.356+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b080075b0 con 0x7f1b1809e900 2024-09-13T17:42:34.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b180a4d80 con 0x7f1b1809e900 2024-09-13T17:42:34.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b14ff9640 1 -- 172.21.15.96:0/2506546457 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1b08019300 con 0x7f1b1809e900 2024-09-13T17:42:34.517 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 msgr2=0x7f1af8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 0x7f1af8037910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1b000ad000 tx=0x7f1b00008040 comp rx=0 tx=0).stop 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 msgr2=0x7f1b1806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800c450 tx=0x7f1b0800c910 comp rx=0 tx=0).stop 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 shutdown_connections 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f1af8035450 0x7f1af8037910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 --2- 172.21.15.96:0/2506546457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809e900 0x7f1b1806f6d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 >> 172.21.15.96:0/2506546457 conn(0x7f1b18066820 msgr2=0x7f1b1812ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 shutdown_connections 2024-09-13T17:42:34.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.516+0000 7f1b1dee8640 1 -- 172.21.15.96:0/2506546457 wait complete. 2024-09-13T17:42:34.535 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:42:34.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:34.536 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:42:34.553 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:42:34.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:42:34.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:34.554 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:42:34.606 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:34.607 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:34.652 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:34.652 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:34.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:34.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.856+0000 7fcc1cc4c640 1 Processor -- start 2024-09-13T17:42:34.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.856+0000 7fcc1cc4c640 1 -- start start 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc1809dc20 0x7fcc181351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1806e130 con 0x7fcc1806f6d0 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc180a6750 con 0x7fcc1809dc20 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc1809dc20 0x7fcc181351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcc1809dc20 0x7fcc181351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48596/0 (socket says 172.21.15.96:48596) 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 -- 172.21.15.96:0/505363455 learned_addr learned my addr 172.21.15.96:0/505363455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 -- 172.21.15.96:0/505363455 >> v1:172.21.15.96:6789/0 conn(0x7fcc1806f6d0 legacy=0x7fcc18134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 -- 172.21.15.96:0/505363455 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc180712e0 con 0x7fcc1809dc20 2024-09-13T17:42:34.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc0dd9b640 1 --2- 172.21.15.96:0/505363455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1809dc20 0x7fcc181351c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000ba90 tx=0x7fcc0001eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c950eb889b30a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc15d9b640 1 -- 172.21.15.96:0/505363455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc000a77f0 con 0x7fcc1809dc20 2024-09-13T17:42:34.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc15d9b640 1 -- 172.21.15.96:0/505363455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc000a7950 con 0x7fcc1809dc20 2024-09-13T17:42:34.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc15d9b640 1 -- 172.21.15.96:0/505363455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc000a7c10 con 0x7fcc1809dc20 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/505363455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1809dc20 msgr2=0x7fcc181351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/505363455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1809dc20 0x7fcc181351c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000ba90 tx=0x7fcc0001eb00 comp rx=0 tx=0).stop 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/505363455 shutdown_connections 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/505363455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1809dc20 0x7fcc181351c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/505363455 >> 172.21.15.96:0/505363455 conn(0x7fcc18098a60 msgr2=0x7fcc1809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/505363455 shutdown_connections 2024-09-13T17:42:34.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.860+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/505363455 wait complete. 2024-09-13T17:42:34.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 Processor -- start 2024-09-13T17:42:34.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 -- start start 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18135570 con 0x7fcc1806f6d0 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48600/0 (socket says 172.21.15.96:48600) 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1659c640 1 -- 172.21.15.96:0/3569967516 learned_addr learned my addr 172.21.15.96:0/3569967516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:34.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1659c640 1 -- 172.21.15.96:0/3569967516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0000b770 con 0x7fcc1806f6d0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1659c640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c3a0 tx=0x7fcc0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc08005150 con 0x7fcc1806f6d0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc080052b0 con 0x7fcc1806f6d0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc180a5f00 con 0x7fcc1806f6d0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcc080067e0 con 0x7fcc1806f6d0 2024-09-13T17:42:34.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1811e030 con 0x7fcc1806f6d0 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcc08006940 con 0x7fcc1806f6d0 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1806f2e0 con 0x7fcc1806f6d0 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 0x7fcbf8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fcc08046b30 con 0x7fcc1806f6d0 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.864+0000 7fcc0dd9b640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 0x7fcbf8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:34.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.868+0000 7fcc0dd9b640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 0x7fcbf8037910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc000b1000 tx=0x7fcc000a9070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:34.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:34.872+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcc08007640 con 0x7fcc1806f6d0 2024-09-13T17:42:35.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.016+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc1811efb0 con 0x7fcc1806f6d0 2024-09-13T17:42:35.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.016+0000 7fcc0f7fe640 1 -- 172.21.15.96:0/3569967516 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fcc080102e0 con 0x7fcc1806f6d0 2024-09-13T17:42:35.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:35.020 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 msgr2=0x7fcbf8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 0x7fcbf8037910 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcc000b1000 tx=0x7fcc000a9070 comp rx=0 tx=0).stop 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 msgr2=0x7fcc1811ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800c3a0 tx=0x7fcc0800c860 comp rx=0 tx=0).stop 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 shutdown_connections 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcbf8035450 0x7fcbf8037910 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 --2- 172.21.15.96:0/3569967516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc1806f6d0 0x7fcc1811ea70 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 >> 172.21.15.96:0/3569967516 conn(0x7fcc18098a60 msgr2=0x7fcc18098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 shutdown_connections 2024-09-13T17:42:35.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.020+0000 7fcc1cc4c640 1 -- 172.21.15.96:0/3569967516 wait complete. 2024-09-13T17:42:35.037 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:35.038 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:35.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.144+0000 7fdcf3bfd640 1 Processor -- start 2024-09-13T17:42:35.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.144+0000 7fdcf3bfd640 1 -- start start 2024-09-13T17:42:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.144+0000 7fdcf3bfd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdcec13af50 0x7fdcec13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.144+0000 7fdcf3bfd640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcec138050 con 0x7fdcec09d690 2024-09-13T17:42:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.144+0000 7fdcf3bfd640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcec09c410 con 0x7fdcec13af50 2024-09-13T17:42:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1999640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdcec09d690 0x7fdcec138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33690/0 (socket says 172.21.15.96:33690) 2024-09-13T17:42:35.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1999640 1 -- 172.21.15.96:0/1420082682 learned_addr learned my addr 172.21.15.96:0/1420082682 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1198640 1 --2- 172.21.15.96:0/1420082682 >> v2:172.21.15.96:3300/0 conn(0x7fdcec13af50 0x7fdcec13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf0997640 1 -- 172.21.15.96:0/1420082682 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272958269 0 0) 0x7fdce8001120 con 0x7fdcec09d690 2024-09-13T17:42:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf0997640 1 -- 172.21.15.96:0/1420082682 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd8003530 con 0x7fdcec09d690 2024-09-13T17:42:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1198640 1 -- 172.21.15.96:0/1420082682 >> v1:172.21.15.96:6789/0 conn(0x7fdcec09d690 legacy=0x7fdcec138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1198640 1 -- 172.21.15.96:0/1420082682 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcec09cba0 con 0x7fdcec13af50 2024-09-13T17:42:35.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf1198640 1 --2- 172.21.15.96:0/1420082682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec13af50 0x7fdcec13d340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009740 tx=0x7fdcdc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6067789b50a1d8ef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf0997640 1 -- 172.21.15.96:0/1420082682 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcdc0a8840 con 0x7fdcec13af50 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf0997640 1 -- 172.21.15.96:0/1420082682 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0a89a0 con 0x7fdcec13af50 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf0997640 1 -- 172.21.15.96:0/1420082682 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcdc0a8c60 con 0x7fdcec13af50 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/1420082682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec13af50 msgr2=0x7fdcec13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/1420082682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec13af50 0x7fdcec13d340 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009740 tx=0x7fdcdc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/1420082682 shutdown_connections 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/1420082682 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec13af50 0x7fdcec13d340 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/1420082682 >> 172.21.15.96:0/1420082682 conn(0x7fdcec06bc60 msgr2=0x7fdcec06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/1420082682 shutdown_connections 2024-09-13T17:42:35.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.148+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/1420082682 wait complete. 2024-09-13T17:42:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 Processor -- start 2024-09-13T17:42:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 -- start start 2024-09-13T17:42:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcec0a43e0 con 0x7fdcec09d690 2024-09-13T17:42:35.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1999640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1999640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48610/0 (socket says 172.21.15.96:48610) 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1999640 1 -- 172.21.15.96:0/2024142894 learned_addr learned my addr 172.21.15.96:0/2024142894 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1999640 1 -- 172.21.15.96:0/2024142894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc009420 con 0x7fdcec09d690 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1999640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c3a0 tx=0x7fdce800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdce8005c20 con 0x7fdcec09d690 2024-09-13T17:42:35.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce8005d80 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec0a61b0 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdce8007200 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcec0a70c0 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fdce8007360 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 0x7fdcd0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fdce80469a0 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcec138050 con 0x7fdcec09d690 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1198640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 0x7fdcd0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.152+0000 7fdcf1198640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 0x7fdcd0037910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0b2000 tx=0x7fdcdc0aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.156+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdcec138050 con 0x7fdcec09d690 2024-09-13T17:42:35.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdcec138050 con 0x7fdcec09d690 2024-09-13T17:42:35.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdce27fc640 1 -- 172.21.15.96:0/2024142894 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdcec138050 con 0x7fdcec09d690 2024-09-13T17:42:35.317 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:35.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 msgr2=0x7fdcd0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 0x7fdcd0037910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0b2000 tx=0x7fdcdc0aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 msgr2=0x7fdcec0a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdce800c3a0 tx=0x7fdce800c860 comp rx=0 tx=0).stop 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 shutdown_connections 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fdcd0035450 0x7fdcd0037910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 --2- 172.21.15.96:0/2024142894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdcec09d690 0x7fdcec0a6b80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 >> 172.21.15.96:0/2024142894 conn(0x7fdcec06bc60 msgr2=0x7fdcec1c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 shutdown_connections 2024-09-13T17:42:35.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.316+0000 7fdcf3bfd640 1 -- 172.21.15.96:0/2024142894 wait complete. 2024-09-13T17:42:35.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:35.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f156527e640 1 Processor -- start 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f156527e640 1 -- start start 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f156527e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f156006ff00 0x7f1560141340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f156527e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15600710a0 con 0x7f156013c4d0 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f156527e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156013c160 con 0x7f156006ff00 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f155e7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f156006ff00 0x7f1560141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f155e7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f156006ff00 0x7f1560141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48624/0 (socket says 172.21.15.96:48624) 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f155e7fc640 1 -- 172.21.15.96:0/3656053864 learned_addr learned my addr 172.21.15.96:0/3656053864 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f155e7fc640 1 -- 172.21.15.96:0/3656053864 >> v1:172.21.15.96:6789/0 conn(0x7f156013c4d0 legacy=0x7f156013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.448+0000 7f155e7fc640 1 -- 172.21.15.96:0/3656053864 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560071bd0 con 0x7f156006ff00 2024-09-13T17:42:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f155e7fc640 1 --2- 172.21.15.96:0/3656053864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f1560141340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15540097f0 tx=0x7f155401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf27bf7d387db61c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f155dffb640 1 -- 172.21.15.96:0/3656053864 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15540ac540 con 0x7f156006ff00 2024-09-13T17:42:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f155dffb640 1 -- 172.21.15.96:0/3656053864 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15540ac6a0 con 0x7f156006ff00 2024-09-13T17:42:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f155dffb640 1 -- 172.21.15.96:0/3656053864 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15540ac960 con 0x7f156006ff00 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- 172.21.15.96:0/3656053864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 msgr2=0x7f1560141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 --2- 172.21.15.96:0/3656053864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f1560141340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f15540097f0 tx=0x7f155401c3f0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- 172.21.15.96:0/3656053864 shutdown_connections 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 --2- 172.21.15.96:0/3656053864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f1560141340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- 172.21.15.96:0/3656053864 >> 172.21.15.96:0/3656053864 conn(0x7f1560098a60 msgr2=0x7f156009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- 172.21.15.96:0/3656053864 shutdown_connections 2024-09-13T17:42:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- 172.21.15.96:0/3656053864 wait complete. 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 Processor -- start 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- start start 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.452+0000 7f156527e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156009d420 con 0x7f156006ff00 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48630/0 (socket says 172.21.15.96:48630) 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155effd640 1 -- 172.21.15.96:0/3346762315 learned_addr learned my addr 172.21.15.96:0/3346762315 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:35.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155effd640 1 -- 172.21.15.96:0/3346762315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540094d0 con 0x7f156006ff00 2024-09-13T17:42:35.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155effd640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1550007fd0 tx=0x7f155000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1550005200 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1550005360 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156013f780 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1550003600 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156013bb30 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f1550003760 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156009d1f0 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 0x7f154c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f1550046c00 con 0x7f156006ff00 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155e7fc640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 0x7f154c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.456+0000 7f155e7fc640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 0x7f154c037910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f15540ae000 tx=0x7f1554008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.460+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1550019d20 con 0x7f156006ff00 2024-09-13T17:42:35.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15600a6fd0 con 0x7f156006ff00 2024-09-13T17:42:35.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f15477fe640 1 -- 172.21.15.96:0/3346762315 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1550017020 con 0x7f156006ff00 2024-09-13T17:42:35.618 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:35.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 msgr2=0x7f154c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 0x7f154c037910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f15540ae000 tx=0x7f1554008040 comp rx=0 tx=0).stop 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 msgr2=0x7f156013d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1550007fd0 tx=0x7f155000c810 comp rx=0 tx=0).stop 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 shutdown_connections 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f154c035450 0x7f154c037910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 --2- 172.21.15.96:0/3346762315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f156006ff00 0x7f156013d940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 >> 172.21.15.96:0/3346762315 conn(0x7f1560098a60 msgr2=0x7f15601359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 shutdown_connections 2024-09-13T17:42:35.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.616+0000 7f156527e640 1 -- 172.21.15.96:0/3346762315 wait complete. 2024-09-13T17:42:35.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:35.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.752+0000 7f7478157640 1 Processor -- start 2024-09-13T17:42:35.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 -- start start 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f747009e820 0x7f747013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74700a68b0 con 0x7f7470134e70 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747009ddc0 con 0x7f747009e820 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7475ef3640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7470134e70 0x7f7470135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33694/0 (socket says 172.21.15.96:33694) 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f74756f2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f747009e820 0x7f747013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7475ef3640 1 -- 172.21.15.96:0/3974829541 learned_addr learned my addr 172.21.15.96:0/3974829541 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:35.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f74756f2640 1 -- 172.21.15.96:0/3974829541 >> v1:172.21.15.96:6789/0 conn(0x7f7470134e70 legacy=0x7f7470135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f74756f2640 1 -- 172.21.15.96:0/3974829541 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470136970 con 0x7f747009e820 2024-09-13T17:42:35.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f74756f2640 1 --2- 172.21.15.96:0/3974829541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f747013ae50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7460009740 tx=0x7f746001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dfd2b5518c26d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7474ef1640 1 -- 172.21.15.96:0/3974829541 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74600ab530 con 0x7f747009e820 2024-09-13T17:42:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7474ef1640 1 -- 172.21.15.96:0/3974829541 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74600a7050 con 0x7f747009e820 2024-09-13T17:42:35.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7474ef1640 1 -- 172.21.15.96:0/3974829541 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74600ab7e0 con 0x7f747009e820 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 -- 172.21.15.96:0/3974829541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 msgr2=0x7f747013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.756+0000 7f7478157640 1 --2- 172.21.15.96:0/3974829541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f747013ae50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7460009740 tx=0x7f746001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/3974829541 shutdown_connections 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 --2- 172.21.15.96:0/3974829541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f747013ae50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/3974829541 >> 172.21.15.96:0/3974829541 conn(0x7f7470098a60 msgr2=0x7f747009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/3974829541 shutdown_connections 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/3974829541 wait complete. 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 Processor -- start 2024-09-13T17:42:35.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- start start 2024-09-13T17:42:35.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74700a3780 con 0x7f747009e820 2024-09-13T17:42:35.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7475ef3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7475ef3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48644/0 (socket says 172.21.15.96:48644) 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7475ef3640 1 -- 172.21.15.96:0/2616869494 learned_addr learned my addr 172.21.15.96:0/2616869494 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7475ef3640 1 -- 172.21.15.96:0/2616869494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009420 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7475ef3640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c480 tx=0x7f746c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c005c20 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c005d80 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747013a5e0 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f746c0071a0 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74700a34c0 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f746c007300 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74700a38c0 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 0x7f7454037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f746c046c20 con 0x7f747009e820 2024-09-13T17:42:35.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74756f2640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 0x7f7454037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:35.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.760+0000 7f74756f2640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 0x7f7454037960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f74600ad000 tx=0x7f7460008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:35.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.764+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f74700a38c0 con 0x7f747009e820 2024-09-13T17:42:35.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.920+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74700a38c0 con 0x7f747009e820 2024-09-13T17:42:35.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f74667fc640 1 -- 172.21.15.96:0/2616869494 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7f74700a38c0 con 0x7f747009e820 2024-09-13T17:42:35.924 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:35.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 msgr2=0x7f7454037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 0x7f7454037960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f74600ad000 tx=0x7f7460008040 comp rx=0 tx=0).stop 2024-09-13T17:42:35.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 msgr2=0x7f74701cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:35.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f746c00c480 tx=0x7f746c00c940 comp rx=0 tx=0).stop 2024-09-13T17:42:35.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 shutdown_connections 2024-09-13T17:42:35.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f74540354a0 0x7f7454037960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 --2- 172.21.15.96:0/2616869494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747009e820 0x7f74701cda00 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:35.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 >> 172.21.15.96:0/2616869494 conn(0x7f7470098a60 msgr2=0x7f7470134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:35.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 shutdown_connections 2024-09-13T17:42:35.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:35.924+0000 7f7478157640 1 -- 172.21.15.96:0/2616869494 wait complete. 2024-09-13T17:42:35.943 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:36.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc798dd4640 1 Processor -- start 2024-09-13T17:42:36.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc798dd4640 1 -- start start 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc798dd4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc79406ff00 0x7fc794141340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc798dd4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940710a0 con 0x7fc79413c4d0 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc798dd4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79413c160 con 0x7fc79406ff00 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc791d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc79406ff00 0x7fc794141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc79259c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc79413c4d0 0x7fc79413c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33702/0 (socket says 172.21.15.96:33702) 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.056+0000 7fc79259c640 1 -- 172.21.15.96:0/3576270890 learned_addr learned my addr 172.21.15.96:0/3576270890 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc791d9b640 1 -- 172.21.15.96:0/3576270890 >> v1:172.21.15.96:6789/0 conn(0x7fc79413c4d0 legacy=0x7fc79413c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc791d9b640 1 -- 172.21.15.96:0/3576270890 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794071bd0 con 0x7fc79406ff00 2024-09-13T17:42:36.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc79159a640 1 -- 172.21.15.96:0/3576270890 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206241779 0 0) 0x7fc7940710a0 con 0x7fc79413c4d0 2024-09-13T17:42:36.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc791d9b640 1 --2- 172.21.15.96:0/3576270890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794141340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1536ab919b9963f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc79159a640 1 -- 172.21.15.96:0/3576270890 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc77c0a57b0 con 0x7fc79406ff00 2024-09-13T17:42:36.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc79159a640 1 -- 172.21.15.96:0/3576270890 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc77c0a5910 con 0x7fc79406ff00 2024-09-13T17:42:36.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc79159a640 1 -- 172.21.15.96:0/3576270890 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc77c0a5bd0 con 0x7fc79406ff00 2024-09-13T17:42:36.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 -- 172.21.15.96:0/3576270890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 msgr2=0x7fc794141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 --2- 172.21.15.96:0/3576270890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794141340 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc77c009d90 tx=0x7fc77c01c880 comp rx=0 tx=0).stop 2024-09-13T17:42:36.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 -- 172.21.15.96:0/3576270890 shutdown_connections 2024-09-13T17:42:36.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 --2- 172.21.15.96:0/3576270890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794141340 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 -- 172.21.15.96:0/3576270890 >> 172.21.15.96:0/3576270890 conn(0x7fc794098a60 msgr2=0x7fc79409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 -- 172.21.15.96:0/3576270890 shutdown_connections 2024-09-13T17:42:36.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.060+0000 7fc798dd4640 1 -- 172.21.15.96:0/3576270890 wait complete. 2024-09-13T17:42:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 Processor -- start 2024-09-13T17:42:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 -- start start 2024-09-13T17:42:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79413f9b0 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc79259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc79259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48668/0 (socket says 172.21.15.96:48668) 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc79259c640 1 -- 172.21.15.96:0/1513026792 learned_addr learned my addr 172.21.15.96:0/1513026792 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc79259c640 1 -- 172.21.15.96:0/1513026792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c009a70 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc79259c640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc78c001ec0 tx=0x7fc78c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc78c005c20 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c005d80 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc79413f100 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc78c007240 con 0x7fc79406ff00 2024-09-13T17:42:36.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7940a6ed0 con 0x7fc79406ff00 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79409a9d0 con 0x7fc79406ff00 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fc78c0073a0 con 0x7fc79406ff00 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 0x7fc774037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fc78c046c40 con 0x7fc79406ff00 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc791d9b640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 0x7fc774037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.064+0000 7fc791d9b640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 0x7fc774037910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0af000 tx=0x7fc77c0a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.068+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc78c019b40 con 0x7fc79406ff00 2024-09-13T17:42:36.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc79413ba20 con 0x7fc79406ff00 2024-09-13T17:42:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc78affd640 1 -- 172.21.15.96:0/1513026792 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc78c017070 con 0x7fc79406ff00 2024-09-13T17:42:36.242 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 msgr2=0x7fc774037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 0x7fc774037910 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0af000 tx=0x7fc77c0a7070 comp rx=0 tx=0).stop 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 msgr2=0x7fc794140000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc78c001ec0 tx=0x7fc78c00c860 comp rx=0 tx=0).stop 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.240+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 shutdown_connections 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.244+0000 7fc798dd4640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc774035450 0x7fc774037910 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.244+0000 7fc798dd4640 1 --2- 172.21.15.96:0/1513026792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc79406ff00 0x7fc794140000 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.244+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 >> 172.21.15.96:0/1513026792 conn(0x7fc794098a60 msgr2=0x7fc794135770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.244+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 shutdown_connections 2024-09-13T17:42:36.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.244+0000 7fc798dd4640 1 -- 172.21.15.96:0/1513026792 wait complete. 2024-09-13T17:42:36.261 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:42:36.261 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:42:36.266 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.424+0000 7f496314c640 1 Processor -- start 2024-09-13T17:42:36.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.424+0000 7f496314c640 1 -- start start 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f495c137490 0x7f495c139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495c071670 con 0x7f495c134670 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495c071360 con 0x7f495c137490 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f495c137490 0x7f495c139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f4960ee8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f495c134670 0x7f495c134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33720/0 (socket says 172.21.15.96:33720) 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f495c137490 0x7f495c139880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48672/0 (socket says 172.21.15.96:48672) 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f4960ee8640 1 -- 172.21.15.96:0/4113751803 learned_addr learned my addr 172.21.15.96:0/4113751803 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495bfff640 1 -- 172.21.15.96:0/4113751803 >> v1:172.21.15.96:6789/0 conn(0x7f495c134670 legacy=0x7f495c134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495b7fe640 1 -- 172.21.15.96:0/4113751803 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714461475 0 0) 0x7f495c071670 con 0x7f495c134670 2024-09-13T17:42:36.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495bfff640 1 -- 172.21.15.96:0/4113751803 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c0a7200 con 0x7f495c137490 2024-09-13T17:42:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495bfff640 1 --2- 172.21.15.96:0/4113751803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c137490 0x7f495c139880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f495000a8e0 tx=0x7f495001d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c674f6e6382a6c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495b7fe640 1 -- 172.21.15.96:0/4113751803 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49500a9550 con 0x7f495c137490 2024-09-13T17:42:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495b7fe640 1 -- 172.21.15.96:0/4113751803 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500a96b0 con 0x7f495c137490 2024-09-13T17:42:36.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f495b7fe640 1 -- 172.21.15.96:0/4113751803 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49500a9970 con 0x7f495c137490 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- 172.21.15.96:0/4113751803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c137490 msgr2=0x7f495c139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 --2- 172.21.15.96:0/4113751803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c137490 0x7f495c139880 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f495000a8e0 tx=0x7f495001d760 comp rx=0 tx=0).stop 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- 172.21.15.96:0/4113751803 shutdown_connections 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 --2- 172.21.15.96:0/4113751803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c137490 0x7f495c139880 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- 172.21.15.96:0/4113751803 >> 172.21.15.96:0/4113751803 conn(0x7f495c098a60 msgr2=0x7f495c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- 172.21.15.96:0/4113751803 shutdown_connections 2024-09-13T17:42:36.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.428+0000 7f496314c640 1 -- 172.21.15.96:0/4113751803 wait complete. 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 Processor -- start 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 -- start start 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c0a0650 con 0x7f495c134670 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f4960ee8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f4960ee8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48674/0 (socket says 172.21.15.96:48674) 2024-09-13T17:42:36.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f4960ee8640 1 -- 172.21.15.96:0/3154546095 learned_addr learned my addr 172.21.15.96:0/3154546095 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:36.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f4960ee8640 1 -- 172.21.15.96:0/3154546095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495000a5c0 con 0x7f495c134670 2024-09-13T17:42:36.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f4960ee8640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c420 tx=0x7f494c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f494c0074d0 con 0x7f495c134670 2024-09-13T17:42:36.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c007630 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f494c004960 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495c09fbf0 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f495c09f8a0 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f494c004ac0 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 0x7f4948037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f495c134a50 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f494c046c60 con 0x7f495c134670 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.432+0000 7f495bfff640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 0x7f4948037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.436+0000 7f495bfff640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 0x7f4948037910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49500b0000 tx=0x7f49500a8000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.440+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f494c01acd0 con 0x7f495c134670 2024-09-13T17:42:36.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.592+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f495c06a1b0 con 0x7f495c134670 2024-09-13T17:42:36.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f49597fa640 1 -- 172.21.15.96:0/3154546095 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f494c018030 con 0x7f495c134670 2024-09-13T17:42:36.598 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:36.598 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:36.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 msgr2=0x7f4948037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 0x7f4948037910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49500b0000 tx=0x7f49500a8000 comp rx=0 tx=0).stop 2024-09-13T17:42:36.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 msgr2=0x7f495c1cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c420 tx=0x7f494c00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 shutdown_connections 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4948035450 0x7f4948037910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 --2- 172.21.15.96:0/3154546095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495c134670 0x7f495c1cb720 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.596+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 >> 172.21.15.96:0/3154546095 conn(0x7f495c098a60 msgr2=0x7f495c12dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.600+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 shutdown_connections 2024-09-13T17:42:36.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.600+0000 7f496314c640 1 -- 172.21.15.96:0/3154546095 wait complete. 2024-09-13T17:42:36.616 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:36.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:36.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb237464640 1 Processor -- start 2024-09-13T17:42:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb237464640 1 -- start start 2024-09-13T17:42:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb237464640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb230137d60 0x7fb2300a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb237464640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300a00b0 con 0x7fb230138140 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb237464640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2300a12d0 con 0x7fb230137d60 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb2349ff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb230138140 0x7fb230138590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33732/0 (socket says 172.21.15.96:33732) 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb2349ff640 1 -- 172.21.15.96:0/101098995 learned_addr learned my addr 172.21.15.96:0/101098995 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb235200640 1 --2- 172.21.15.96:0/101098995 >> v2:172.21.15.96:3300/0 conn(0x7fb230137d60 0x7fb2300a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb227fff640 1 -- 172.21.15.96:0/101098995 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1652653595 0 0) 0x7fb2300a00b0 con 0x7fb230138140 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb227fff640 1 -- 172.21.15.96:0/101098995 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c003530 con 0x7fb230138140 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb235200640 1 -- 172.21.15.96:0/101098995 >> v1:172.21.15.96:6789/0 conn(0x7fb230138140 legacy=0x7fb230138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb235200640 1 -- 172.21.15.96:0/101098995 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300a0350 con 0x7fb230137d60 2024-09-13T17:42:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb235200640 1 --2- 172.21.15.96:0/101098995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a0d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009740 tx=0x7fb22c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e91272a3d3bc00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb227fff640 1 -- 172.21.15.96:0/101098995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb22c0ab530 con 0x7fb230137d60 2024-09-13T17:42:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb227fff640 1 -- 172.21.15.96:0/101098995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c0a7050 con 0x7fb230137d60 2024-09-13T17:42:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.732+0000 7fb227fff640 1 -- 172.21.15.96:0/101098995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb22c0ab7c0 con 0x7fb230137d60 2024-09-13T17:42:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/101098995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 msgr2=0x7fb2300a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 --2- 172.21.15.96:0/101098995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a0d90 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb22c009740 tx=0x7fb22c01c390 comp rx=0 tx=0).stop 2024-09-13T17:42:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/101098995 shutdown_connections 2024-09-13T17:42:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 --2- 172.21.15.96:0/101098995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a0d90 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/101098995 >> 172.21.15.96:0/101098995 conn(0x7fb23006ef80 msgr2=0x7fb2300713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/101098995 shutdown_connections 2024-09-13T17:42:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/101098995 wait complete. 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 Processor -- start 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- start start 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2300a3da0 con 0x7fb230137d60 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb235200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb235200640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48700/0 (socket says 172.21.15.96:48700) 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb235200640 1 -- 172.21.15.96:0/2026417842 learned_addr learned my addr 172.21.15.96:0/2026417842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb235200640 1 -- 172.21.15.96:0/2026417842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c009420 con 0x7fb230137d60 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb235200640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0ae040 tx=0x7fb22c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb22c0b7450 con 0x7fb230137d60 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c0b75b0 con 0x7fb230137d60 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb22c0b7850 con 0x7fb230137d60 2024-09-13T17:42:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300a5d70 con 0x7fb230137d60 2024-09-13T17:42:36.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2300a4df0 con 0x7fb230137d60 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300a3280 con 0x7fb230137d60 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb22c0b79b0 con 0x7fb230137d60 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 0x7fb214037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.736+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb22c0e25e0 con 0x7fb230137d60 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.740+0000 7fb2349ff640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 0x7fb214037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.740+0000 7fb2349ff640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 0x7fb214037960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb220007fd0 tx=0x7fb220007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:36.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.740+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb22c0bf9a0 con 0x7fb230137d60 2024-09-13T17:42:36.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2300a3280 con 0x7fb230137d60 2024-09-13T17:42:36.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb225ffb640 1 -- 172.21.15.96:0/2026417842 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2300a3280 con 0x7fb230137d60 2024-09-13T17:42:36.889 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 msgr2=0x7fb214037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 0x7fb214037960 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb220007fd0 tx=0x7fb220007980 comp rx=0 tx=0).stop 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 msgr2=0x7fb2300a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0ae040 tx=0x7fb22c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 shutdown_connections 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb2140354a0 0x7fb214037960 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 --2- 172.21.15.96:0/2026417842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb230137d60 0x7fb2300a5830 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 >> 172.21.15.96:0/2026417842 conn(0x7fb23006ef80 msgr2=0x7fb230134ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 shutdown_connections 2024-09-13T17:42:36.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:36.888+0000 7fb237464640 1 -- 172.21.15.96:0/2026417842 wait complete. 2024-09-13T17:42:36.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.024+0000 7f2cdcb18640 1 Processor -- start 2024-09-13T17:42:37.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.024+0000 7f2cdcb18640 1 -- start start 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2cd809e820 0x7f2cd813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd80a68b0 con 0x7f2cd8134e70 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd809ddc0 con 0x7f2cd809e820 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2cd8134e70 0x7f2cd8135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33746/0 (socket says 172.21.15.96:33746) 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd659c640 1 -- 172.21.15.96:0/154031632 learned_addr learned my addr 172.21.15.96:0/154031632 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:37.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd5d9b640 1 --2- 172.21.15.96:0/154031632 >> v2:172.21.15.96:3300/0 conn(0x7f2cd809e820 0x7f2cd813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd559a640 1 -- 172.21.15.96:0/154031632 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026313418 0 0) 0x7f2cd80a68b0 con 0x7f2cd8134e70 2024-09-13T17:42:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd559a640 1 -- 172.21.15.96:0/154031632 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc4003530 con 0x7f2cd8134e70 2024-09-13T17:42:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd5d9b640 1 -- 172.21.15.96:0/154031632 >> v1:172.21.15.96:6789/0 conn(0x7f2cd8134e70 legacy=0x7f2cd8135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd5d9b640 1 -- 172.21.15.96:0/154031632 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8136970 con 0x7f2cd809e820 2024-09-13T17:42:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd5d9b640 1 --2- 172.21.15.96:0/154031632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd813ae50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009740 tx=0x7f2cc001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58302e3425cabc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd559a640 1 -- 172.21.15.96:0/154031632 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc00ab530 con 0x7f2cd809e820 2024-09-13T17:42:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd559a640 1 -- 172.21.15.96:0/154031632 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc00a7050 con 0x7f2cd809e820 2024-09-13T17:42:37.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cd559a640 1 -- 172.21.15.96:0/154031632 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc00ab7c0 con 0x7f2cd809e820 2024-09-13T17:42:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- 172.21.15.96:0/154031632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 msgr2=0x7f2cd813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/154031632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd813ae50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009740 tx=0x7f2cc001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- 172.21.15.96:0/154031632 shutdown_connections 2024-09-13T17:42:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/154031632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd813ae50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- 172.21.15.96:0/154031632 >> 172.21.15.96:0/154031632 conn(0x7f2cd8098a60 msgr2=0x7f2cd809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- 172.21.15.96:0/154031632 shutdown_connections 2024-09-13T17:42:37.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.028+0000 7f2cdcb18640 1 -- 172.21.15.96:0/154031632 wait complete. 2024-09-13T17:42:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 Processor -- start 2024-09-13T17:42:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 -- start start 2024-09-13T17:42:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd811e1b0 con 0x7f2cd809e820 2024-09-13T17:42:37.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48726/0 (socket says 172.21.15.96:48726) 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd659c640 1 -- 172.21.15.96:0/793567296 learned_addr learned my addr 172.21.15.96:0/793567296 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd659c640 1 -- 172.21.15.96:0/793567296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc0009420 con 0x7f2cd809e820 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd659c640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc8005c20 con 0x7f2cd809e820 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8005d80 con 0x7f2cd809e820 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd811e3c0 con 0x7f2cd809e820 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cc80071a0 con 0x7f2cd809e820 2024-09-13T17:42:37.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd80a58b0 con 0x7f2cd809e820 2024-09-13T17:42:37.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd80a6190 con 0x7f2cd809e820 2024-09-13T17:42:37.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2cc8007300 con 0x7f2cd809e820 2024-09-13T17:42:37.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 0x7f2cb8037940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f2cc8047470 con 0x7f2cd809e820 2024-09-13T17:42:37.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.032+0000 7f2cd5d9b640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 0x7f2cb8037940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.036+0000 7f2cd5d9b640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 0x7f2cb8037940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00ad000 tx=0x7f2cc00a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.036+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2cc8010b60 con 0x7f2cd809e820 2024-09-13T17:42:37.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2cd806a1b0 con 0x7f2cd809e820 2024-09-13T17:42:37.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cceffd640 1 -- 172.21.15.96:0/793567296 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2cc8017030 con 0x7f2cd809e820 2024-09-13T17:42:37.194 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 msgr2=0x7f2cb8037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 0x7f2cb8037940 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00ad000 tx=0x7f2cc00a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 msgr2=0x7f2cd80a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).stop 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 shutdown_connections 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2cb8035480 0x7f2cb8037940 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 --2- 172.21.15.96:0/793567296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2cd809e820 0x7f2cd80a3aa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.192+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 >> 172.21.15.96:0/793567296 conn(0x7f2cd8098a60 msgr2=0x7f2cd8134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.196+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 shutdown_connections 2024-09-13T17:42:37.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.196+0000 7f2cdcb18640 1 -- 172.21.15.96:0/793567296 wait complete. 2024-09-13T17:42:37.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:37.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5dd7de640 1 Processor -- start 2024-09-13T17:42:37.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5dd7de640 1 -- start start 2024-09-13T17:42:37.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5dd7de640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc5d8134e70 0x7fc5d809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5dd7de640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d809ed60 con 0x7fc5d8135250 2024-09-13T17:42:37.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5dd7de640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d809ddc0 con 0x7fc5d8134e70 2024-09-13T17:42:37.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5d6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc5d8134e70 0x7fc5d809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5d67fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc5d8135250 0x7fc5d8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33762/0 (socket says 172.21.15.96:33762) 2024-09-13T17:42:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5d6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc5d8134e70 0x7fc5d809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48742/0 (socket says 172.21.15.96:48742) 2024-09-13T17:42:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.320+0000 7fc5d67fc640 1 -- 172.21.15.96:0/2234783028 learned_addr learned my addr 172.21.15.96:0/2234783028 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d6ffd640 1 -- 172.21.15.96:0/2234783028 >> v1:172.21.15.96:6789/0 conn(0x7fc5d8135250 legacy=0x7fc5d8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d5ffb640 1 -- 172.21.15.96:0/2234783028 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457910412 0 0) 0x7fc5c8001120 con 0x7fc5d8135250 2024-09-13T17:42:37.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d6ffd640 1 -- 172.21.15.96:0/2234783028 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80a68b0 con 0x7fc5d8134e70 2024-09-13T17:42:37.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d6ffd640 1 --2- 172.21.15.96:0/2234783028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d809e820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009740 tx=0x7fc5cc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19b1d1cd6bf56d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d5ffb640 1 -- 172.21.15.96:0/2234783028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc0ab530 con 0x7fc5d8134e70 2024-09-13T17:42:37.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d5ffb640 1 -- 172.21.15.96:0/2234783028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0a7050 con 0x7fc5d8134e70 2024-09-13T17:42:37.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5d5ffb640 1 -- 172.21.15.96:0/2234783028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc0ab7e0 con 0x7fc5d8134e70 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 -- 172.21.15.96:0/2234783028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 msgr2=0x7fc5d809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/2234783028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d809e820 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009740 tx=0x7fc5cc01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 -- 172.21.15.96:0/2234783028 shutdown_connections 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/2234783028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d809e820 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 -- 172.21.15.96:0/2234783028 >> 172.21.15.96:0/2234783028 conn(0x7fc5d8098a60 msgr2=0x7fc5d809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 -- 172.21.15.96:0/2234783028 shutdown_connections 2024-09-13T17:42:37.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.324+0000 7fc5dd7de640 1 -- 172.21.15.96:0/2234783028 wait complete. 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 Processor -- start 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 -- start start 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80a3cc0 con 0x7fc5d8134e70 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48744/0 (socket says 172.21.15.96:48744) 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d6ffd640 1 -- 172.21.15.96:0/3427971063 learned_addr learned my addr 172.21.15.96:0/3427971063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d6ffd640 1 -- 172.21.15.96:0/3427971063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc009420 con 0x7fc5d8134e70 2024-09-13T17:42:37.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d6ffd640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc019030 tx=0x7fc5cc01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc0b7450 con 0x7fc5d8134e70 2024-09-13T17:42:37.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0b75b0 con 0x7fc5d8134e70 2024-09-13T17:42:37.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d8132f70 con 0x7fc5d8134e70 2024-09-13T17:42:37.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5cc0b7890 con 0x7fc5d8134e70 2024-09-13T17:42:37.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d80a3690 con 0x7fc5d8134e70 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fc5cc0b79f0 con 0x7fc5d8134e70 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d8071150 con 0x7fc5d8134e70 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 0x7fc5c4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fc5cc0e2000 con 0x7fc5d8134e70 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d67fc640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 0x7fc5c4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.328+0000 7fc5d67fc640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 0x7fc5c4037910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c10 tx=0x7fc5c8001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.332+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5cc0c74e0 con 0x7fc5d8134e70 2024-09-13T17:42:37.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5d81368b0 con 0x7fc5d8134e70 2024-09-13T17:42:37.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5bf7fe640 1 -- 172.21.15.96:0/3427971063 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5cc0b0020 con 0x7fc5d8134e70 2024-09-13T17:42:37.494 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 msgr2=0x7fc5c4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 0x7fc5c4037910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8000c10 tx=0x7fc5c8001320 comp rx=0 tx=0).stop 2024-09-13T17:42:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 msgr2=0x7fc5d806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.492+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc019030 tx=0x7fc5cc01ac60 comp rx=0 tx=0).stop 2024-09-13T17:42:37.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 shutdown_connections 2024-09-13T17:42:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc5c4035450 0x7fc5c4037910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 --2- 172.21.15.96:0/3427971063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5d8134e70 0x7fc5d806f6d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 >> 172.21.15.96:0/3427971063 conn(0x7fc5d8098a60 msgr2=0x7fc5d8098f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 shutdown_connections 2024-09-13T17:42:37.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.496+0000 7fc5dd7de640 1 -- 172.21.15.96:0/3427971063 wait complete. 2024-09-13T17:42:37.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f54a1fb6640 1 Processor -- start 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f54a1fb6640 1 -- start start 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f54a1fb6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f549c06ed40 0x7f549c13a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f54a1fb6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c071670 con 0x7f549c134670 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f54a1fb6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c071360 con 0x7f549c06ed40 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f549c134670 0x7f549c134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33778/0 (socket says 172.21.15.96:33778) 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549b7fe640 1 -- 172.21.15.96:0/3831143798 learned_addr learned my addr 172.21.15.96:0/3831143798 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549affd640 1 --2- 172.21.15.96:0/3831143798 >> v2:172.21.15.96:3300/0 conn(0x7f549c06ed40 0x7f549c13a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549a7fc640 1 -- 172.21.15.96:0/3831143798 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206319871 0 0) 0x7f549c071670 con 0x7f549c134670 2024-09-13T17:42:37.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549a7fc640 1 -- 172.21.15.96:0/3831143798 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5484003530 con 0x7f549c134670 2024-09-13T17:42:37.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549affd640 1 -- 172.21.15.96:0/3831143798 >> v1:172.21.15.96:6789/0 conn(0x7f549c134670 legacy=0x7f549c134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.636+0000 7f549affd640 1 -- 172.21.15.96:0/3831143798 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0a7200 con 0x7f549c06ed40 2024-09-13T17:42:37.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549affd640 1 --2- 172.21.15.96:0/3831143798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c13a5b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5490009740 tx=0x7f549001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1a4246cd84e273 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549a7fc640 1 -- 172.21.15.96:0/3831143798 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54900ab530 con 0x7f549c06ed40 2024-09-13T17:42:37.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549a7fc640 1 -- 172.21.15.96:0/3831143798 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54900a7050 con 0x7f549c06ed40 2024-09-13T17:42:37.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549a7fc640 1 -- 172.21.15.96:0/3831143798 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54900ab7e0 con 0x7f549c06ed40 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3831143798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 msgr2=0x7f549c13a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3831143798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c13a5b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5490009740 tx=0x7f549001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3831143798 shutdown_connections 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3831143798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c13a5b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3831143798 >> 172.21.15.96:0/3831143798 conn(0x7f549c098a60 msgr2=0x7f549c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3831143798 shutdown_connections 2024-09-13T17:42:37.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3831143798 wait complete. 2024-09-13T17:42:37.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 Processor -- start 2024-09-13T17:42:37.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- start start 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f54a1fb6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0a1460 con 0x7f549c06ed40 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48758/0 (socket says 172.21.15.96:48758) 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.640+0000 7f549b7fe640 1 -- 172.21.15.96:0/3428408967 learned_addr learned my addr 172.21.15.96:0/3428408967 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f549b7fe640 1 -- 172.21.15.96:0/3428408967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009420 con 0x7f549c06ed40 2024-09-13T17:42:37.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f549b7fe640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c470 tx=0x7f548c00c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f548c005c20 con 0x7f549c06ed40 2024-09-13T17:42:37.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c005d80 con 0x7f549c06ed40 2024-09-13T17:42:37.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0a2300 con 0x7f549c06ed40 2024-09-13T17:42:37.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f548c0071e0 con 0x7f549c06ed40 2024-09-13T17:42:37.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c133370 con 0x7f549c06ed40 2024-09-13T17:42:37.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c09d500 con 0x7f549c06ed40 2024-09-13T17:42:37.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f548c007340 con 0x7f549c06ed40 2024-09-13T17:42:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 0x7f5488037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f548c046f30 con 0x7f549c06ed40 2024-09-13T17:42:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f549affd640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 0x7f5488037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:37.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.644+0000 7f549affd640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 0x7f5488037910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54900ad000 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:37.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.648+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f548c01b3b0 con 0x7f549c06ed40 2024-09-13T17:42:37.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.804+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f549c09d500 con 0x7f549c06ed40 2024-09-13T17:42:37.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.804+0000 7f5483fff640 1 -- 172.21.15.96:0/3428408967 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v15) v1 ==== 80+0+0 (secure 0 0 0) 0x7f549c09d500 con 0x7f549c06ed40 2024-09-13T17:42:37.807 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:42:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 msgr2=0x7f5488037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 0x7f5488037910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f54900ad000 tx=0x7f5490008040 comp rx=0 tx=0).stop 2024-09-13T17:42:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 msgr2=0x7f549c0a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c470 tx=0x7f548c00c930 comp rx=0 tx=0).stop 2024-09-13T17:42:37.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 shutdown_connections 2024-09-13T17:42:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f5488035450 0x7f5488037910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 --2- 172.21.15.96:0/3428408967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f549c06ed40 0x7f549c0a1bb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 >> 172.21.15.96:0/3428408967 conn(0x7f549c098a60 msgr2=0x7f549c09a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 shutdown_connections 2024-09-13T17:42:37.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:37.808+0000 7f54a1fb6640 1 -- 172.21.15.96:0/3428408967 wait complete. 2024-09-13T17:42:37.824 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:42:37.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:37.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:42:37.836 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:42:37.837 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:42:37.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:42:37.837 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:42:37.885 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-09-13T17:42:37.885 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:38.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.040+0000 7f0e66c79640 1 Processor -- start 2024-09-13T17:42:38.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.040+0000 7f0e66c79640 1 -- start start 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e66c79640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e60134e70 0x7f0e6009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e66c79640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e6009ed60 con 0x7f0e60135250 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e66c79640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e6009ddc0 con 0x7f0e60134e70 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e64a15640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0e60134e70 0x7f0e6009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e5ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0e60135250 0x7f0e60135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33792/0 (socket says 172.21.15.96:33792) 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e5ffff640 1 -- 172.21.15.96:0/3334550187 learned_addr learned my addr 172.21.15.96:0/3334550187 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e64a15640 1 -- 172.21.15.96:0/3334550187 >> v1:172.21.15.96:6789/0 conn(0x7f0e60135250 legacy=0x7f0e60135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e64a15640 1 -- 172.21.15.96:0/3334550187 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600a68b0 con 0x7f0e60134e70 2024-09-13T17:42:38.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.044+0000 7f0e5f7fe640 1 -- 172.21.15.96:0/3334550187 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4029767859 0 0) 0x7f0e48001120 con 0x7f0e60135250 2024-09-13T17:42:38.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 --2- 172.21.15.96:0/3334550187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6009e820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009740 tx=0x7f0e5001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df510b400632b7b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e5f7fe640 1 -- 172.21.15.96:0/3334550187 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500ab530 con 0x7f0e60134e70 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e5f7fe640 1 -- 172.21.15.96:0/3334550187 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500a7050 con 0x7f0e60134e70 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e5f7fe640 1 -- 172.21.15.96:0/3334550187 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500ab7e0 con 0x7f0e60134e70 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- 172.21.15.96:0/3334550187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 msgr2=0x7f0e6009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 --2- 172.21.15.96:0/3334550187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6009e820 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009740 tx=0x7f0e5001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- 172.21.15.96:0/3334550187 shutdown_connections 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 --2- 172.21.15.96:0/3334550187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6009e820 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- 172.21.15.96:0/3334550187 >> 172.21.15.96:0/3334550187 conn(0x7f0e60098a60 msgr2=0x7f0e6009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- 172.21.15.96:0/3334550187 shutdown_connections 2024-09-13T17:42:38.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- 172.21.15.96:0/3334550187 wait complete. 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 Processor -- start 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- start start 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e66c79640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60070770 con 0x7f0e60134e70 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48788/0 (socket says 172.21.15.96:48788) 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 -- 172.21.15.96:0/2484267783 learned_addr learned my addr 172.21.15.96:0/2484267783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 -- 172.21.15.96:0/2484267783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50009420 con 0x7f0e60134e70 2024-09-13T17:42:38.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.048+0000 7f0e64a15640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0e50019030 tx=0x7f0e5001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500abde0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50004030 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600a44a0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e500042d0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600a55c0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0e50003620 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e601368b0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 0x7f0e40037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f0e500e23f0 con 0x7f0e60134e70 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5ffff640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 0x7f0e40037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.052+0000 7f0e5ffff640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 0x7f0e40037910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c10 tx=0x7f0e48001320 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.056+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0e50003e40 con 0x7f0e60134e70 2024-09-13T17:42:38.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e600a4630 con 0x7f0e60134e70 2024-09-13T17:42:38.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e5d7fa640 1 -- 172.21.15.96:0/2484267783 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f0e500b0030 con 0x7f0e60134e70 2024-09-13T17:42:38.238 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:38.238 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:38.238 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 msgr2=0x7f0e40037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 0x7f0e40037910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0e48000c10 tx=0x7f0e48001320 comp rx=0 tx=0).stop 2024-09-13T17:42:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 msgr2=0x7f0e6006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0e50019030 tx=0x7f0e5001ac60 comp rx=0 tx=0).stop 2024-09-13T17:42:38.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 shutdown_connections 2024-09-13T17:42:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0e40035450 0x7f0e40037910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 --2- 172.21.15.96:0/2484267783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e60134e70 0x7f0e6006f6d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 >> 172.21.15.96:0/2484267783 conn(0x7f0e60098a60 msgr2=0x7f0e60098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.236+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 shutdown_connections 2024-09-13T17:42:38.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.240+0000 7f0e66c79640 1 -- 172.21.15.96:0/2484267783 wait complete. 2024-09-13T17:42:38.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:38.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.364+0000 7f02b8652640 1 Processor -- start 2024-09-13T17:42:38.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.364+0000 7f02b8652640 1 -- start start 2024-09-13T17:42:38.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f02b0134e90 0x7f02b0071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02b0070de0 con 0x7f02b0135270 2024-09-13T17:42:38.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02b0071d60 con 0x7f02b0134e90 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b63ee640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f02b0134e90 0x7f02b0071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b5bed640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f02b0135270 0x7f02b006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33794/0 (socket says 172.21.15.96:33794) 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b63ee640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f02b0134e90 0x7f02b0071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48804/0 (socket says 172.21.15.96:48804) 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b5bed640 1 -- 172.21.15.96:0/120591895 learned_addr learned my addr 172.21.15.96:0/120591895 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b63ee640 1 -- 172.21.15.96:0/120591895 >> v1:172.21.15.96:6789/0 conn(0x7f02b0135270 legacy=0x7f02b006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b53ec640 1 -- 172.21.15.96:0/120591895 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179412679 0 0) 0x7f02b0070de0 con 0x7f02b0135270 2024-09-13T17:42:38.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b63ee640 1 -- 172.21.15.96:0/120591895 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b006f110 con 0x7f02b0134e90 2024-09-13T17:42:38.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b63ee640 1 --2- 172.21.15.96:0/120591895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b0071530 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009740 tx=0x7f02a001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c245ef8f71d0c419 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b53ec640 1 -- 172.21.15.96:0/120591895 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02a00ab530 con 0x7f02b0134e90 2024-09-13T17:42:38.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b53ec640 1 -- 172.21.15.96:0/120591895 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a00a7050 con 0x7f02b0134e90 2024-09-13T17:42:38.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b53ec640 1 -- 172.21.15.96:0/120591895 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02a00ab7e0 con 0x7f02b0134e90 2024-09-13T17:42:38.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- 172.21.15.96:0/120591895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 msgr2=0x7f02b0071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 --2- 172.21.15.96:0/120591895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b0071530 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009740 tx=0x7f02a001c390 comp rx=0 tx=0).stop 2024-09-13T17:42:38.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- 172.21.15.96:0/120591895 shutdown_connections 2024-09-13T17:42:38.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 --2- 172.21.15.96:0/120591895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b0071530 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- 172.21.15.96:0/120591895 >> 172.21.15.96:0/120591895 conn(0x7f02b0098a60 msgr2=0x7f02b009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- 172.21.15.96:0/120591895 shutdown_connections 2024-09-13T17:42:38.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.368+0000 7f02b8652640 1 -- 172.21.15.96:0/120591895 wait complete. 2024-09-13T17:42:38.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 Processor -- start 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 -- start start 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b0133960 con 0x7f02b0134e90 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b63ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b63ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48820/0 (socket says 172.21.15.96:48820) 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b63ee640 1 -- 172.21.15.96:0/4123124144 learned_addr learned my addr 172.21.15.96:0/4123124144 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b63ee640 1 -- 172.21.15.96:0/4123124144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a0009420 con 0x7f02b0134e90 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b63ee640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f02a0019fc0 tx=0x7f02a001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02a00abea0 con 0x7f02b0134e90 2024-09-13T17:42:38.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a0004020 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b00a6010 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02a00042c0 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b00a55d0 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b011d2a0 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f02a00b2050 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 0x7f0294037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f02a00e19f0 con 0x7f02b0134e90 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b5bed640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 0x7f0294037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.372+0000 7f02b5bed640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 0x7f0294037910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02ac007fd0 tx=0x7f02ac007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.380+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f02a00b2300 con 0x7f02b0134e90 2024-09-13T17:42:38.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.524+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02b006a1b0 con 0x7f02b0134e90 2024-09-13T17:42:38.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.524+0000 7f02a6ffd640 1 -- 172.21.15.96:0/4123124144 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f02a00b0030 con 0x7f02b0134e90 2024-09-13T17:42:38.528 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:38.528 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 msgr2=0x7f0294037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 0x7f0294037910 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f02ac007fd0 tx=0x7f02ac007980 comp rx=0 tx=0).stop 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 msgr2=0x7f02b01cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f02a0019fc0 tx=0x7f02a001abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 shutdown_connections 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0294035450 0x7f0294037910 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 --2- 172.21.15.96:0/4123124144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f02b0134e90 0x7f02b01cc8c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 >> 172.21.15.96:0/4123124144 conn(0x7f02b0098a60 msgr2=0x7f02b0133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 shutdown_connections 2024-09-13T17:42:38.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.528+0000 7f02b8652640 1 -- 172.21.15.96:0/4123124144 wait complete. 2024-09-13T17:42:38.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:00.679081+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:01.283445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3600000143051147, 'score_stable': 1.3600000143051147, 'optimal_score': 1, 'raw_score_acting': 1.3600000143051147, 'raw_score_stable': 1.3600000143051147, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:38.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:38.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.656+0000 7f081a296640 1 Processor -- start 2024-09-13T17:42:38.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- start start 2024-09-13T17:42:38.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f081409d690 0x7f0814138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0814138050 con 0x7f08141387d0 2024-09-13T17:42:38.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081409c410 con 0x7f081409d690 2024-09-13T17:42:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f0813fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f081409d690 0x7f0814138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f080b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f08141387d0 0x7f0814134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33808/0 (socket says 172.21.15.96:33808) 2024-09-13T17:42:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f080b7fe640 1 -- 172.21.15.96:0/1153487449 learned_addr learned my addr 172.21.15.96:0/1153487449 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f0813fff640 1 -- 172.21.15.96:0/1153487449 >> v1:172.21.15.96:6789/0 conn(0x7f08141387d0 legacy=0x7f0814134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f0813fff640 1 -- 172.21.15.96:0/1153487449 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081409cba0 con 0x7f081409d690 2024-09-13T17:42:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f0813fff640 1 --2- 172.21.15.96:0/1153487449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f0814138290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0804009740 tx=0x7f080401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4820fb01ba24ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f08137fe640 1 -- 172.21.15.96:0/1153487449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08040a8840 con 0x7f081409d690 2024-09-13T17:42:38.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f08137fe640 1 -- 172.21.15.96:0/1153487449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040a89a0 con 0x7f081409d690 2024-09-13T17:42:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f08137fe640 1 -- 172.21.15.96:0/1153487449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08040a8c60 con 0x7f081409d690 2024-09-13T17:42:38.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- 172.21.15.96:0/1153487449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 msgr2=0x7f0814138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 --2- 172.21.15.96:0/1153487449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f0814138290 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0804009740 tx=0x7f080401ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- 172.21.15.96:0/1153487449 shutdown_connections 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 --2- 172.21.15.96:0/1153487449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f0814138290 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- 172.21.15.96:0/1153487449 >> 172.21.15.96:0/1153487449 conn(0x7f081406bc60 msgr2=0x7f081406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- 172.21.15.96:0/1153487449 shutdown_connections 2024-09-13T17:42:38.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.660+0000 7f081a296640 1 -- 172.21.15.96:0/1153487449 wait complete. 2024-09-13T17:42:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 Processor -- start 2024-09-13T17:42:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 -- start start 2024-09-13T17:42:38.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08140a5e10 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f0813fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f0813fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48844/0 (socket says 172.21.15.96:48844) 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f0813fff640 1 -- 172.21.15.96:0/2181714591 learned_addr learned my addr 172.21.15.96:0/2181714591 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f0813fff640 1 -- 172.21.15.96:0/2181714591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0804009420 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f0813fff640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f080401c840 tx=0x7f080401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08040b0840 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08040b09a0 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08140a70a0 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f08040b0c60 con 0x7f081409d690 2024-09-13T17:42:38.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08140a4300 con 0x7f081409d690 2024-09-13T17:42:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081409bc00 con 0x7f081409d690 2024-09-13T17:42:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f08040b7040 con 0x7f081409d690 2024-09-13T17:42:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 0x7f07f4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f08040e5350 con 0x7f081409d690 2024-09-13T17:42:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f080b7fe640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 0x7f07f4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.664+0000 7f080b7fe640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 0x7f07f4037910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f07fc005890 tx=0x7f07fc007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.668+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f081409bc00 con 0x7f081409d690 2024-09-13T17:42:38.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f081409bc00 con 0x7f081409d690 2024-09-13T17:42:38.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f08117fa640 1 -- 172.21.15.96:0/2181714591 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f081409bc00 con 0x7f081409d690 2024-09-13T17:42:38.822 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:38.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 msgr2=0x7f07f4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f081a296640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 0x7f07f4037910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f07fc005890 tx=0x7f07fc007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 msgr2=0x7f08140a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.820+0000 7f081a296640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f080401c840 tx=0x7f080401b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 shutdown_connections 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f07f4035450 0x7f07f4037910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 --2- 172.21.15.96:0/2181714591 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f081409d690 0x7f08140a6b60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 >> 172.21.15.96:0/2181714591 conn(0x7f081406bc60 msgr2=0x7f08141bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 shutdown_connections 2024-09-13T17:42:38.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.824+0000 7f081a296640 1 -- 172.21.15.96:0/2181714591 wait complete. 2024-09-13T17:42:38.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:42:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7dedea4640 1 Processor -- start 2024-09-13T17:42:38.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7dedea4640 1 -- start start 2024-09-13T17:42:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7dedea4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7de8138850 0x7f7de80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7dedea4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de80a00b0 con 0x7f7de8137d60 2024-09-13T17:42:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7dedea4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de80a12d0 con 0x7f7de8138850 2024-09-13T17:42:38.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7de6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7de8138850 0x7f7de80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7de6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7de8138850 0x7f7de80a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48860/0 (socket says 172.21.15.96:48860) 2024-09-13T17:42:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7de6ffd640 1 -- 172.21.15.96:0/1997330320 learned_addr learned my addr 172.21.15.96:0/1997330320 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.948+0000 7f7de77fe640 1 --1- 172.21.15.96:0/1997330320 >> v1:172.21.15.96:6789/0 conn(0x7f7de8137d60 0x7f7de8138140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33818/0 (socket says 172.21.15.96:33818) 2024-09-13T17:42:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de6ffd640 1 -- 172.21.15.96:0/1997330320 >> v1:172.21.15.96:6789/0 conn(0x7f7de8137d60 legacy=0x7f7de8138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de6ffd640 1 -- 172.21.15.96:0/1997330320 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de80a0350 con 0x7f7de8138850 2024-09-13T17:42:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de6ffd640 1 --2- 172.21.15.96:0/1997330320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8138850 0x7f7de80a0d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097f0 tx=0x7f7dd801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ddd7f2e3a22baf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de67fc640 1 -- 172.21.15.96:0/1997330320 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd80ab530 con 0x7f7de8138850 2024-09-13T17:42:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de67fc640 1 -- 172.21.15.96:0/1997330320 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd80a7050 con 0x7f7de8138850 2024-09-13T17:42:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7de67fc640 1 -- 172.21.15.96:0/1997330320 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd80ab7e0 con 0x7f7de8138850 2024-09-13T17:42:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- 172.21.15.96:0/1997330320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8138850 msgr2=0x7f7de80a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 --2- 172.21.15.96:0/1997330320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8138850 0x7f7de80a0d90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80097f0 tx=0x7f7dd801c450 comp rx=0 tx=0).stop 2024-09-13T17:42:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- 172.21.15.96:0/1997330320 shutdown_connections 2024-09-13T17:42:38.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 --2- 172.21.15.96:0/1997330320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8138850 0x7f7de80a0d90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:38.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- 172.21.15.96:0/1997330320 >> 172.21.15.96:0/1997330320 conn(0x7f7de806ef80 msgr2=0x7f7de80713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:38.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- 172.21.15.96:0/1997330320 shutdown_connections 2024-09-13T17:42:38.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- 172.21.15.96:0/1997330320 wait complete. 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 Processor -- start 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.952+0000 7f7dedea4640 1 -- start start 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7dedea4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7dedea4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de80a8300 con 0x7f7de8137d60 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48870/0 (socket says 172.21.15.96:48870) 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de77fe640 1 -- 172.21.15.96:0/465623663 learned_addr learned my addr 172.21.15.96:0/465623663 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de77fe640 1 -- 172.21.15.96:0/465623663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd80094d0 con 0x7f7de8137d60 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de77fe640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c410 tx=0x7f7dd000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd0007410 con 0x7f7de8137d60 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0007570 con 0x7f7de8137d60 2024-09-13T17:42:38.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7dd00047f0 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de80a75a0 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de813c4a0 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f7dd0004950 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de80a0d90 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 0x7f7dc8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f7dd00470e0 con 0x7f7de8137d60 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de6ffd640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 0x7f7dc8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:38.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.956+0000 7f7de6ffd640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 0x7f7dc8037910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad000 tx=0x7f7dd80a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:38.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:38.960+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7dd00192c0 con 0x7f7de8137d60 2024-09-13T17:42:39.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.112+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7de80a5100 con 0x7f7de8137d60 2024-09-13T17:42:39.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.112+0000 7f7ddffff640 1 -- 172.21.15.96:0/465623663 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v15) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7dd001aac0 con 0x7f7de8137d60 2024-09-13T17:42:39.115 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:42:39.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 msgr2=0x7f7dc8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 0x7f7dc8037910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80ad000 tx=0x7f7dd80a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:39.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 msgr2=0x7f7de80a29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000c410 tx=0x7f7dd000c8d0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 shutdown_connections 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7dc8035450 0x7f7dc8037910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 --2- 172.21.15.96:0/465623663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7de8137d60 0x7f7de80a29c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 >> 172.21.15.96:0/465623663 conn(0x7f7de806ef80 msgr2=0x7f7de8137340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 shutdown_connections 2024-09-13T17:42:39.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.116+0000 7f7dedea4640 1 -- 172.21.15.96:0/465623663 wait complete. 2024-09-13T17:42:39.132 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:42:39.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82cd29640 1 Processor -- start 2024-09-13T17:42:39.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82cd29640 1 -- start start 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82cd29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe82813af50 0x7fe82813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82cd29640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe828138050 con 0x7fe82809d690 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82cd29640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82809c410 con 0x7fe82813af50 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe82813af50 0x7fe82813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe82813af50 0x7fe82813d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48874/0 (socket says 172.21.15.96:48874) 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 -- 172.21.15.96:0/862545788 learned_addr learned my addr 172.21.15.96:0/862545788 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe82659c640 1 --1- 172.21.15.96:0/862545788 >> v1:172.21.15.96:6789/0 conn(0x7fe82809d690 0x7fe828138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33820/0 (socket says 172.21.15.96:33820) 2024-09-13T17:42:39.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 -- 172.21.15.96:0/862545788 >> v1:172.21.15.96:6789/0 conn(0x7fe82809d690 legacy=0x7fe828138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 -- 172.21.15.96:0/862545788 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82809cba0 con 0x7fe82813af50 2024-09-13T17:42:39.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe825d9b640 1 -- 172.21.15.96:0/862545788 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569539904 0 0) 0x7fe818001120 con 0x7fe82809d690 2024-09-13T17:42:39.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe81ffff640 1 --2- 172.21.15.96:0/862545788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82813af50 0x7fe82813d340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe8100097f0 tx=0x7fe81001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e2d78329dc028b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe825d9b640 1 -- 172.21.15.96:0/862545788 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8100a8900 con 0x7fe82813af50 2024-09-13T17:42:39.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe825d9b640 1 -- 172.21.15.96:0/862545788 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8100a8a60 con 0x7fe82813af50 2024-09-13T17:42:39.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.248+0000 7fe825d9b640 1 -- 172.21.15.96:0/862545788 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8100a8d20 con 0x7fe82813af50 2024-09-13T17:42:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/862545788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82813af50 msgr2=0x7fe82813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 --2- 172.21.15.96:0/862545788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82813af50 0x7fe82813d340 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe8100097f0 tx=0x7fe81001ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/862545788 shutdown_connections 2024-09-13T17:42:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 --2- 172.21.15.96:0/862545788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82813af50 0x7fe82813d340 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/862545788 >> 172.21.15.96:0/862545788 conn(0x7fe82806bc60 msgr2=0x7fe82806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/862545788 shutdown_connections 2024-09-13T17:42:39.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/862545788 wait complete. 2024-09-13T17:42:39.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 Processor -- start 2024-09-13T17:42:39.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- start start 2024-09-13T17:42:39.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8280a5c90 con 0x7fe82809d690 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48878/0 (socket says 172.21.15.96:48878) 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82659c640 1 -- 172.21.15.96:0/3684285212 learned_addr learned my addr 172.21.15.96:0/3684285212 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82659c640 1 -- 172.21.15.96:0/3684285212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100094d0 con 0x7fe82809d690 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82659c640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c390 tx=0x7fe81800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe818007410 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe818007590 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8280a6f20 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8180047f0 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe828137400 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82809bc00 con 0x7fe82809d690 2024-09-13T17:42:39.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe818004950 con 0x7fe82809d690 2024-09-13T17:42:39.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 0x7fe808037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.252+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe8180472b0 con 0x7fe82809d690 2024-09-13T17:42:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.256+0000 7fe81ffff640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 0x7fe808037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.256+0000 7fe81ffff640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 0x7fe808037910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8100b2000 tx=0x7fe8100aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.260+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe82809bc00 con 0x7fe82809d690 2024-09-13T17:42:39.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.404+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe828138050 con 0x7fe82809d690 2024-09-13T17:42:39.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.404+0000 7fe81effd640 1 -- 172.21.15.96:0/3684285212 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v15) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe828138050 con 0x7fe82809d690 2024-09-13T17:42:39.407 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:42:39.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 msgr2=0x7fe808037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 0x7fe808037910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8100b2000 tx=0x7fe8100aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 msgr2=0x7fe8280a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe81800c390 tx=0x7fe81800c850 comp rx=0 tx=0).stop 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 shutdown_connections 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe808035450 0x7fe808037910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 --2- 172.21.15.96:0/3684285212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe82809d690 0x7fe8280a69e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 >> 172.21.15.96:0/3684285212 conn(0x7fe82806bc60 msgr2=0x7fe82809f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 shutdown_connections 2024-09-13T17:42:39.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.408+0000 7fe82cd29640 1 -- 172.21.15.96:0/3684285212 wait complete. 2024-09-13T17:42:39.426 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:39.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.528+0000 7f9c89242640 1 Processor -- start 2024-09-13T17:42:39.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.528+0000 7f9c89242640 1 -- start start 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9c84134e70 0x7f9c8409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8409ed60 con 0x7f9c84135250 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8409ddc0 con 0x7f9c84134e70 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c82d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9c84134e70 0x7f9c8409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c8259c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9c84135250 0x7f9c84135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33830/0 (socket says 172.21.15.96:33830) 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c8259c640 1 -- 172.21.15.96:0/399255822 learned_addr learned my addr 172.21.15.96:0/399255822 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c81d9b640 1 -- 172.21.15.96:0/399255822 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761021812 0 0) 0x7f9c740014a0 con 0x7f9c84135250 2024-09-13T17:42:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c81d9b640 1 -- 172.21.15.96:0/399255822 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c70003530 con 0x7f9c84135250 2024-09-13T17:42:39.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c82d9d640 1 -- 172.21.15.96:0/399255822 >> v1:172.21.15.96:6789/0 conn(0x7f9c84135250 legacy=0x7f9c84135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c82d9d640 1 -- 172.21.15.96:0/399255822 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c840a68b0 con 0x7f9c84134e70 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c82d9d640 1 --2- 172.21.15.96:0/399255822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c8409e820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009740 tx=0x7f9c6c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886329eaab1f74ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c81d9b640 1 -- 172.21.15.96:0/399255822 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c6c0ab530 con 0x7f9c84134e70 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c81d9b640 1 -- 172.21.15.96:0/399255822 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c0a7050 con 0x7f9c84134e70 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c81d9b640 1 -- 172.21.15.96:0/399255822 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c6c0ab7e0 con 0x7f9c84134e70 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- 172.21.15.96:0/399255822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 msgr2=0x7f9c8409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 --2- 172.21.15.96:0/399255822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c8409e820 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009740 tx=0x7f9c6c01c420 comp rx=0 tx=0).stop 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- 172.21.15.96:0/399255822 shutdown_connections 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 --2- 172.21.15.96:0/399255822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c8409e820 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- 172.21.15.96:0/399255822 >> 172.21.15.96:0/399255822 conn(0x7f9c84098a60 msgr2=0x7f9c8409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- 172.21.15.96:0/399255822 shutdown_connections 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.532+0000 7f9c89242640 1 -- 172.21.15.96:0/399255822 wait complete. 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 Processor -- start 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 -- start start 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c840a60d0 con 0x7f9c84134e70 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c82d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c82d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48896/0 (socket says 172.21.15.96:48896) 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c82d9d640 1 -- 172.21.15.96:0/3144988982 learned_addr learned my addr 172.21.15.96:0/3144988982 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c82d9d640 1 -- 172.21.15.96:0/3144988982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c009420 con 0x7f9c84134e70 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c82d9d640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c01a050 tx=0x7f9c6c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c6c0abdf0 con 0x7f9c84134e70 2024-09-13T17:42:39.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6c0b4410 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c8411e350 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c6c0b46b0 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8409ab80 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8409a720 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f9c6c0b2040 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 0x7f9c64037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9c6c0e1680 con 0x7f9c84134e70 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c8259c640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 0x7f9c64037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.536+0000 7f9c8259c640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 0x7f9c64037910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c740020b0 tx=0x7f9c740014e0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.540+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9c6c0bc3b0 con 0x7f9c84134e70 2024-09-13T17:42:39.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.728+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c841368b0 con 0x7f9c84134e70 2024-09-13T17:42:39.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.728+0000 7f9c7b7fe640 1 -- 172.21.15.96:0/3144988982 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f9c6c0b0030 con 0x7f9c84134e70 2024-09-13T17:42:39.730 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:39.730 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:39.731 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:39.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 msgr2=0x7f9c64037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 0x7f9c64037910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c740020b0 tx=0x7f9c740014e0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 msgr2=0x7f9c841320a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c01a050 tx=0x7f9c6c0191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 shutdown_connections 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f9c64035450 0x7f9c64037910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 --2- 172.21.15.96:0/3144988982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c84134e70 0x7f9c841320a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 >> 172.21.15.96:0/3144988982 conn(0x7f9c84098a60 msgr2=0x7f9c84133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 shutdown_connections 2024-09-13T17:42:39.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.732+0000 7f9c89242640 1 -- 172.21.15.96:0/3144988982 wait complete. 2024-09-13T17:42:39.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.856+0000 7fc381bfd640 1 Processor -- start 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.856+0000 7fc381bfd640 1 -- start start 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc37c138080 0x7fc37c138460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc37c0a2290 con 0x7fc37c1389a0 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc37c06a8d0 con 0x7fc37c138080 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc37c138080 0x7fc37c138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc37c138080 0x7fc37c138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48910/0 (socket says 172.21.15.96:48910) 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37affd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc37c1389a0 0x7fc37c09ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33844/0 (socket says 172.21.15.96:33844) 2024-09-13T17:42:39.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 -- 172.21.15.96:0/1191723461 learned_addr learned my addr 172.21.15.96:0/1191723461 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:39.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 -- 172.21.15.96:0/1191723461 >> v1:172.21.15.96:6789/0 conn(0x7fc37c1389a0 legacy=0x7fc37c09ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 -- 172.21.15.96:0/1191723461 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c0a2e00 con 0x7fc37c138080 2024-09-13T17:42:39.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37b7fe640 1 --2- 172.21.15.96:0/1191723461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c138460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc3640097f0 tx=0x7fc36401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51fc91a1f1c3d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37a7fc640 1 -- 172.21.15.96:0/1191723461 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3640ab530 con 0x7fc37c138080 2024-09-13T17:42:39.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37a7fc640 1 -- 172.21.15.96:0/1191723461 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3640a7050 con 0x7fc37c138080 2024-09-13T17:42:39.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc37a7fc640 1 -- 172.21.15.96:0/1191723461 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3640ab7e0 con 0x7fc37c138080 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- 172.21.15.96:0/1191723461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 msgr2=0x7fc37c138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 --2- 172.21.15.96:0/1191723461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c138460 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc3640097f0 tx=0x7fc36401c450 comp rx=0 tx=0).stop 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- 172.21.15.96:0/1191723461 shutdown_connections 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 --2- 172.21.15.96:0/1191723461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c138460 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- 172.21.15.96:0/1191723461 >> 172.21.15.96:0/1191723461 conn(0x7fc37c0a7390 msgr2=0x7fc37c0a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- 172.21.15.96:0/1191723461 shutdown_connections 2024-09-13T17:42:39.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.860+0000 7fc381bfd640 1 -- 172.21.15.96:0/1191723461 wait complete. 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 Processor -- start 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 -- start start 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37c071b70 con 0x7fc37c138080 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48926/0 (socket says 172.21.15.96:48926) 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37b7fe640 1 -- 172.21.15.96:0/2491579018 learned_addr learned my addr 172.21.15.96:0/2491579018 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37b7fe640 1 -- 172.21.15.96:0/2491579018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3640094d0 con 0x7fc37c138080 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37b7fe640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc36401a080 tx=0x7fc36401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3640b6630 con 0x7fc37c138080 2024-09-13T17:42:39.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3640b6790 con 0x7fc37c138080 2024-09-13T17:42:39.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3640b6a50 con 0x7fc37c138080 2024-09-13T17:42:39.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37c071d60 con 0x7fc37c138080 2024-09-13T17:42:39.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37c06f7f0 con 0x7fc37c138080 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc37c06fe20 con 0x7fc37c138080 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fc3640b2040 con 0x7fc37c138080 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 0x7fc35c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fc3640e1e40 con 0x7fc37c138080 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37affd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 0x7fc35c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:39.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.864+0000 7fc37affd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 0x7fc35c037910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0a2d30 tx=0x7fc36c007d90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:39.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:39.868+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3640bd950 con 0x7fc37c138080 2024-09-13T17:42:40.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.016+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc37c13d100 con 0x7fc37c138080 2024-09-13T17:42:40.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc373fff640 1 -- 172.21.15.96:0/2491579018 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fc3640b0030 con 0x7fc37c138080 2024-09-13T17:42:40.022 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:40.022 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 msgr2=0x7fc35c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 0x7fc35c037910 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc37c0a2d30 tx=0x7fc36c007d90 comp rx=0 tx=0).stop 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 msgr2=0x7fc37c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc36401a080 tx=0x7fc36401ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 shutdown_connections 2024-09-13T17:42:40.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fc35c035450 0x7fc35c037910 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 --2- 172.21.15.96:0/2491579018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c138080 0x7fc37c06ec10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 >> 172.21.15.96:0/2491579018 conn(0x7fc37c0a7390 msgr2=0x7fc37c137290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 shutdown_connections 2024-09-13T17:42:40.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.020+0000 7fc381bfd640 1 -- 172.21.15.96:0/2491579018 wait complete. 2024-09-13T17:42:40.040 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:42:40.040 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:40.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558fa12640 1 Processor -- start 2024-09-13T17:42:40.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558fa12640 1 -- start start 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558fa12640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5588134e70 0x7f558809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558fa12640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558809ed60 con 0x7f5588135250 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558fa12640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558809ddc0 con 0x7f5588134e70 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558d7ae640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5588134e70 0x7f558809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558d7ae640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5588134e70 0x7f558809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48940/0 (socket says 172.21.15.96:48940) 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558d7ae640 1 -- 172.21.15.96:0/1156510994 learned_addr learned my addr 172.21.15.96:0/1156510994 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558cfad640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5588135250 0x7f5588135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33860/0 (socket says 172.21.15.96:33860) 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558d7ae640 1 -- 172.21.15.96:0/1156510994 >> v1:172.21.15.96:6789/0 conn(0x7f5588135250 legacy=0x7f5588135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.152+0000 7f558d7ae640 1 -- 172.21.15.96:0/1156510994 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880a68b0 con 0x7f5588134e70 2024-09-13T17:42:40.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 --2- 172.21.15.96:0/1156510994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f558809e820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f55840097f0 tx=0x7f558401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb52af7555de2da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f557bfff640 1 -- 172.21.15.96:0/1156510994 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55840ac540 con 0x7f5588134e70 2024-09-13T17:42:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f557bfff640 1 -- 172.21.15.96:0/1156510994 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55840ac6a0 con 0x7f5588134e70 2024-09-13T17:42:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f557bfff640 1 -- 172.21.15.96:0/1156510994 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55840ac960 con 0x7f5588134e70 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/1156510994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 msgr2=0x7f558809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 --2- 172.21.15.96:0/1156510994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f558809e820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f55840097f0 tx=0x7f558401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/1156510994 shutdown_connections 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 --2- 172.21.15.96:0/1156510994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f558809e820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/1156510994 >> 172.21.15.96:0/1156510994 conn(0x7f5588098a60 msgr2=0x7f558809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/1156510994 shutdown_connections 2024-09-13T17:42:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/1156510994 wait complete. 2024-09-13T17:42:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 Processor -- start 2024-09-13T17:42:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- start start 2024-09-13T17:42:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558811e140 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48954/0 (socket says 172.21.15.96:48954) 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 -- 172.21.15.96:0/3147011729 learned_addr learned my addr 172.21.15.96:0/3147011729 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 -- 172.21.15.96:0/3147011729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840094d0 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558d7ae640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55840190f0 tx=0x7f5584019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55840bb070 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55840b7470 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558811d750 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55840b7730 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880a58f0 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55881368b0 con 0x7f5588134e70 2024-09-13T17:42:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f55840b3070 con 0x7f5588134e70 2024-09-13T17:42:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.156+0000 7f5579ffb640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 0x7f556c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.160+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55840e17a0 con 0x7f5588134e70 2024-09-13T17:42:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.160+0000 7f558cfad640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 0x7f556c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.160+0000 7f558cfad640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 0x7f556c037910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f557c001600 tx=0x7f557c007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.164+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f55840b4380 con 0x7f5588134e70 2024-09-13T17:42:40.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.340+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f558809a7c0 con 0x7f5588134e70 2024-09-13T17:42:40.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.340+0000 7f5579ffb640 1 -- 172.21.15.96:0/3147011729 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7f55840b1070 con 0x7f5588134e70 2024-09-13T17:42:40.343 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:40.343 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:40.344 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 msgr2=0x7f556c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 0x7f556c037910 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f557c001600 tx=0x7f557c007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 msgr2=0x7f55880a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55840190f0 tx=0x7f5584019280 comp rx=0 tx=0).stop 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 shutdown_connections 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f556c035450 0x7f556c037910 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 --2- 172.21.15.96:0/3147011729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5588134e70 0x7f55880a3a40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 >> 172.21.15.96:0/3147011729 conn(0x7f5588098a60 msgr2=0x7f5588134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 shutdown_connections 2024-09-13T17:42:40.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.344+0000 7f558fa12640 1 -- 172.21.15.96:0/3147011729 wait complete. 2024-09-13T17:42:40.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.464+0000 7fd6cff1d640 1 Processor -- start 2024-09-13T17:42:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.464+0000 7fd6cff1d640 1 -- start start 2024-09-13T17:42:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd6c8135270 0x7fd6c813cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c8070de0 con 0x7fd6c8134e90 2024-09-13T17:42:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c8071d60 con 0x7fd6c8135270 2024-09-13T17:42:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cd4b8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd6c8135270 0x7fd6c813cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cdcb9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd6c8134e90 0x7fd6c8071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33872/0 (socket says 172.21.15.96:33872) 2024-09-13T17:42:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cdcb9640 1 -- 172.21.15.96:0/2058509384 learned_addr learned my addr 172.21.15.96:0/2058509384 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cd4b8640 1 -- 172.21.15.96:0/2058509384 >> v1:172.21.15.96:6789/0 conn(0x7fd6c8134e90 legacy=0x7fd6c8071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cd4b8640 1 -- 172.21.15.96:0/2058509384 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c806f110 con 0x7fd6c8135270 2024-09-13T17:42:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cd4b8640 1 --2- 172.21.15.96:0/2058509384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8135270 0x7fd6c813cfe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c401c770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0180877b34a5adc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cccb7640 1 -- 172.21.15.96:0/2058509384 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6c40a9570 con 0x7fd6c8135270 2024-09-13T17:42:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cccb7640 1 -- 172.21.15.96:0/2058509384 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c40a96d0 con 0x7fd6c8135270 2024-09-13T17:42:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cccb7640 1 -- 172.21.15.96:0/2058509384 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6c40a9990 con 0x7fd6c8135270 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2058509384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8135270 msgr2=0x7fd6c813cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2058509384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8135270 0x7fd6c813cfe0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4009d90 tx=0x7fd6c401c770 comp rx=0 tx=0).stop 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2058509384 shutdown_connections 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2058509384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8135270 0x7fd6c813cfe0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2058509384 >> 172.21.15.96:0/2058509384 conn(0x7fd6c8098a60 msgr2=0x7fd6c809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2058509384 shutdown_connections 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.468+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2058509384 wait complete. 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 Processor -- start 2024-09-13T17:42:40.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 -- start start 2024-09-13T17:42:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80a6110 con 0x7fd6c8134e90 2024-09-13T17:42:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cdcb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cdcb9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48972/0 (socket says 172.21.15.96:48972) 2024-09-13T17:42:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cdcb9640 1 -- 172.21.15.96:0/2253960451 learned_addr learned my addr 172.21.15.96:0/2253960451 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cdcb9640 1 -- 172.21.15.96:0/2253960451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c4009a70 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cdcb9640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c700 tx=0x7fd6bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6bc007200 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc007380 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c811d690 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6bc007660 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80a5400 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c8136c60 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd6bc0047f0 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 0x7fd6ac037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fd6bc046eb0 con 0x7fd6c8134e90 2024-09-13T17:42:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cd4b8640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 0x7fd6ac037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.472+0000 7fd6cd4b8640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 0x7fd6ac037910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40af000 tx=0x7fd6c40a7000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.476+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd6bc011be0 con 0x7fd6c8134e90 2024-09-13T17:42:40.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.656+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6c8071530 con 0x7fd6c8134e90 2024-09-13T17:42:40.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.656+0000 7fd6ba7fc640 1 -- 172.21.15.96:0/2253960451 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3210 (secure 0 0 0) 0x7fd6bc018020 con 0x7fd6c8134e90 2024-09-13T17:42:40.661 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:40.661 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6828/3843232757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3843232757},{"type":"v1","addr":"172.21.15.96:6828","nonce":3843232757}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:02.313336+0000","modified":"2024-09-13T17:42:04.319061+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4145},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4145":{"gid":4145,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/45340133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6827","nonce":45340133},{"type":"v1","addr":"172.21.15.96:6829","nonce":45340133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-09-13T17:42:40.661 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-09-13T17:42:40.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 msgr2=0x7fd6ac037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 0x7fd6ac037910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd6c40af000 tx=0x7fd6c40a7000 comp rx=0 tx=0).stop 2024-09-13T17:42:40.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 msgr2=0x7fd6c811dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c700 tx=0x7fd6bc00cbc0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 shutdown_connections 2024-09-13T17:42:40.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd6ac035450 0x7fd6ac037910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 --2- 172.21.15.96:0/2253960451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c8134e90 0x7fd6c811dde0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 >> 172.21.15.96:0/2253960451 conn(0x7fd6c8098a60 msgr2=0x7fd6c8137170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 shutdown_connections 2024-09-13T17:42:40.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.660+0000 7fd6cff1d640 1 -- 172.21.15.96:0/2253960451 wait complete. 2024-09-13T17:42:40.677 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:40.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.792+0000 7fe6dc8d4640 1 Processor -- start 2024-09-13T17:42:40.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.792+0000 7fe6dc8d4640 1 -- start start 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6d809d690 0x7fe6d8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d8138050 con 0x7fe6d81387d0 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d809c410 con 0x7fe6d809d690 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6d809d690 0x7fe6d8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6d809d690 0x7fe6d8138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48988/0 (socket says 172.21.15.96:48988) 2024-09-13T17:42:40.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 -- 172.21.15.96:0/1508476051 learned_addr learned my addr 172.21.15.96:0/1508476051 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 -- 172.21.15.96:0/1508476051 >> v1:172.21.15.96:6789/0 conn(0x7fe6d81387d0 legacy=0x7fe6d8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 -- 172.21.15.96:0/1508476051 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d809cba0 con 0x7fe6d809d690 2024-09-13T17:42:40.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d559a640 1 -- 172.21.15.96:0/1508476051 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463954615 0 0) 0x7fe6cc001120 con 0x7fe6d81387d0 2024-09-13T17:42:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d659c640 1 --2- 172.21.15.96:0/1508476051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d8138290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80097f0 tx=0x7fe6c801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f067196c3fd7c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d559a640 1 -- 172.21.15.96:0/1508476051 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6c80a8900 con 0x7fe6d809d690 2024-09-13T17:42:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d559a640 1 -- 172.21.15.96:0/1508476051 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80a8a60 con 0x7fe6d809d690 2024-09-13T17:42:40.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6d559a640 1 -- 172.21.15.96:0/1508476051 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6c80a8d20 con 0x7fe6d809d690 2024-09-13T17:42:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/1508476051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 msgr2=0x7fe6d8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/1508476051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d8138290 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6c80097f0 tx=0x7fe6c801ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/1508476051 shutdown_connections 2024-09-13T17:42:40.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/1508476051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d8138290 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/1508476051 >> 172.21.15.96:0/1508476051 conn(0x7fe6d806bc60 msgr2=0x7fe6d806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/1508476051 shutdown_connections 2024-09-13T17:42:40.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.796+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/1508476051 wait complete. 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 Processor -- start 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 -- start start 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d80a61a0 con 0x7fe6d809d690 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48998/0 (socket says 172.21.15.96:48998) 2024-09-13T17:42:40.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d659c640 1 -- 172.21.15.96:0/2732738248 learned_addr learned my addr 172.21.15.96:0/2732738248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:40.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d659c640 1 -- 172.21.15.96:0/2732738248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c80094d0 con 0x7fe6d809d690 2024-09-13T17:42:40.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d659c640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe6c801b8e0 tx=0x7fe6c801d510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6c8004150 con 0x7fe6d809d690 2024-09-13T17:42:40.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c80042b0 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d80a5ee0 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6c80b0710 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d809f420 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fe6c80b7040 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d809bc00 con 0x7fe6d809d690 2024-09-13T17:42:40.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 0x7fe6b8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:40.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe6c80e5380 con 0x7fe6d809d690 2024-09-13T17:42:40.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.800+0000 7fe6d5d9b640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 0x7fe6b8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:40.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.804+0000 7fe6d5d9b640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 0x7fe6b8037910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc001380 tx=0x7fe6cc0077f0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:40.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.808+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6d809bc00 con 0x7fe6d809d690 2024-09-13T17:42:40.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.956+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6d80a62e0 con 0x7fe6d809d690 2024-09-13T17:42:40.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.956+0000 7fe6c6ffd640 1 -- 172.21.15.96:0/2732738248 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fe6d80a62e0 con 0x7fe6d809d690 2024-09-13T17:42:40.959 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:40.959 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":15,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:03.314979+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:00.679081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:01.283445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3600000143051147,"score_stable":1.3600000143051147,"optimal_score":1,"raw_score_acting":1.3600000143051147,"raw_score_stable":1.3600000143051147,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 msgr2=0x7fe6b8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 0x7fe6b8037910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc001380 tx=0x7fe6cc0077f0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 msgr2=0x7fe6d80a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe6c801b8e0 tx=0x7fe6c801d510 comp rx=0 tx=0).stop 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 shutdown_connections 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fe6b8035450 0x7fe6b8037910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 --2- 172.21.15.96:0/2732738248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6d809d690 0x7fe6d80a6920 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:40.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 >> 172.21.15.96:0/2732738248 conn(0x7fe6d806bc60 msgr2=0x7fe6d809e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:40.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 shutdown_connections 2024-09-13T17:42:40.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:40.960+0000 7fe6dc8d4640 1 -- 172.21.15.96:0/2732738248 wait complete. 2024-09-13T17:42:40.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:42:41.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 Processor -- start 2024-09-13T17:42:41.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 -- start start 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb4c009e820 0x7fb4c013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00a68b0 con 0x7fb4c0134e70 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c009ddc0 con 0x7fb4c009e820 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb4c009e820 0x7fb4c013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb4c009e820 0x7fb4c013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49002/0 (socket says 172.21.15.96:49002) 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 -- 172.21.15.96:0/1619928384 learned_addr learned my addr 172.21.15.96:0/1619928384 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4beffd640 1 --1- 172.21.15.96:0/1619928384 >> v1:172.21.15.96:6789/0 conn(0x7fb4c0134e70 0x7fb4c0135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33896/0 (socket says 172.21.15.96:33896) 2024-09-13T17:42:41.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 -- 172.21.15.96:0/1619928384 >> v1:172.21.15.96:6789/0 conn(0x7fb4c0134e70 legacy=0x7fb4c0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 -- 172.21.15.96:0/1619928384 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0136970 con 0x7fb4c009e820 2024-09-13T17:42:41.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4be7fc640 1 --2- 172.21.15.96:0/1619928384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c013ae50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80097f0 tx=0x7fb4a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d509ab98879e0e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4bdffb640 1 -- 172.21.15.96:0/1619928384 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a80ac540 con 0x7fb4c009e820 2024-09-13T17:42:41.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4bdffb640 1 -- 172.21.15.96:0/1619928384 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a80ac6a0 con 0x7fb4c009e820 2024-09-13T17:42:41.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4bdffb640 1 -- 172.21.15.96:0/1619928384 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4a80ac960 con 0x7fb4c009e820 2024-09-13T17:42:41.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 -- 172.21.15.96:0/1619928384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 msgr2=0x7fb4c013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.084+0000 7fb4c55da640 1 --2- 172.21.15.96:0/1619928384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c013ae50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80097f0 tx=0x7fb4a801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/1619928384 shutdown_connections 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 --2- 172.21.15.96:0/1619928384 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c013ae50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/1619928384 >> 172.21.15.96:0/1619928384 conn(0x7fb4c0098a60 msgr2=0x7fb4c009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/1619928384 shutdown_connections 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/1619928384 wait complete. 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 Processor -- start 2024-09-13T17:42:41.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- start start 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c013a670 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49004/0 (socket says 172.21.15.96:49004) 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4beffd640 1 -- 172.21.15.96:0/2033076462 learned_addr learned my addr 172.21.15.96:0/2033076462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4beffd640 1 -- 172.21.15.96:0/2033076462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a80094d0 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4beffd640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c3a0 tx=0x7fb4b000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4b0007410 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b0007570 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c00a5600 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4b00047f0 con 0x7fb4c009e820 2024-09-13T17:42:41.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c011c930 con 0x7fb4c009e820 2024-09-13T17:42:41.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fb4b0004950 con 0x7fb4c009e820 2024-09-13T17:42:41.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c00a52f0 con 0x7fb4c009e820 2024-09-13T17:42:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 0x7fb4a0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fb4b00470f0 con 0x7fb4c009e820 2024-09-13T17:42:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.088+0000 7fb4be7fc640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 0x7fb4a0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.092+0000 7fb4be7fc640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 0x7fb4a0037910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80ae000 tx=0x7fb4a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.096+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb4b0004e30 con 0x7fb4c009e820 2024-09-13T17:42:41.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.280+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb4c011cb80 con 0x7fb4c009e820 2024-09-13T17:42:41.285 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:42:41.284+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:42:41.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.292+0000 7fb4b77fe640 1 -- 172.21.15.96:0/2033076462 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb4b001ab00 con 0x7fb4c009e820 2024-09-13T17:42:41.294 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.292+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 msgr2=0x7fb4a0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.292+0000 7fb4c55da640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 0x7fb4a0037910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80ae000 tx=0x7fb4a8008040 comp rx=0 tx=0).stop 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.292+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 msgr2=0x7fb4c012f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.292+0000 7fb4c55da640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000c3a0 tx=0x7fb4b000c860 comp rx=0 tx=0).stop 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 shutdown_connections 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fb4a0035450 0x7fb4a0037910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 --2- 172.21.15.96:0/2033076462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4c009e820 0x7fb4c012f000 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 >> 172.21.15.96:0/2033076462 conn(0x7fb4c0098a60 msgr2=0x7fb4c0099610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 shutdown_connections 2024-09-13T17:42:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.296+0000 7fb4c55da640 1 -- 172.21.15.96:0/2033076462 wait complete. 2024-09-13T17:42:41.298 INFO:tasks.ceph.mds.a.smithi096.stderr: -1722> 2024-09-13T17:42:06.274+0000 7f2853a7f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:42:41.313 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:42:41.334 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bcee92640 1 Processor -- start 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bcee92640 1 -- start start 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bcee92640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bc813af50 0x7f2bc813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bcee92640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc8138050 con 0x7f2bc809d690 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bcee92640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc809c410 con 0x7f2bc813af50 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bc813af50 0x7f2bc813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bc813af50 0x7f2bc813d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49024/0 (socket says 172.21.15.96:49024) 2024-09-13T17:42:41.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 -- 172.21.15.96:0/1251278096 learned_addr learned my addr 172.21.15.96:0/1251278096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:41.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bccc2e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2bc809d690 0x7f2bc8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33914/0 (socket says 172.21.15.96:33914) 2024-09-13T17:42:41.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 -- 172.21.15.96:0/1251278096 >> v1:172.21.15.96:6789/0 conn(0x7f2bc809d690 legacy=0x7f2bc8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 -- 172.21.15.96:0/1251278096 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc809cba0 con 0x7f2bc813af50 2024-09-13T17:42:41.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc7fff640 1 --2- 172.21.15.96:0/1251278096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc813af50 0x7f2bc813d340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009e50 tx=0x7f2bb801f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f6ef0753b4b996 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.416+0000 7f2bc77fe640 1 -- 172.21.15.96:0/1251278096 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb8009200 con 0x7f2bc813af50 2024-09-13T17:42:41.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc77fe640 1 -- 172.21.15.96:0/1251278096 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80ab050 con 0x7f2bc813af50 2024-09-13T17:42:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc77fe640 1 -- 172.21.15.96:0/1251278096 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bb80af550 con 0x7f2bc813af50 2024-09-13T17:42:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/1251278096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc813af50 msgr2=0x7f2bc813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:41.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 --2- 172.21.15.96:0/1251278096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc813af50 0x7f2bc813d340 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009e50 tx=0x7f2bb801f210 comp rx=0 tx=0).stop 2024-09-13T17:42:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/1251278096 shutdown_connections 2024-09-13T17:42:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 --2- 172.21.15.96:0/1251278096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc813af50 0x7f2bc813d340 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/1251278096 >> 172.21.15.96:0/1251278096 conn(0x7f2bc806bc60 msgr2=0x7f2bc806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/1251278096 shutdown_connections 2024-09-13T17:42:41.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/1251278096 wait complete. 2024-09-13T17:42:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 Processor -- start 2024-09-13T17:42:41.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- start start 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc80a2be0 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bccc2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bccc2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49034/0 (socket says 172.21.15.96:49034) 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bccc2e640 1 -- 172.21.15.96:0/2802881070 learned_addr learned my addr 172.21.15.96:0/2802881070 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bccc2e640 1 -- 172.21.15.96:0/2802881070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8009b30 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bccc2e640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c3b0 tx=0x7f2bc000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bc0007410 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0007570 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc80a51a0 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bc00047f0 con 0x7f2bc809d690 2024-09-13T17:42:41.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc80a35d0 con 0x7f2bc809d690 2024-09-13T17:42:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc809da70 con 0x7f2bc809d690 2024-09-13T17:42:41.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.420+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f2bc0004950 con 0x7f2bc809d690 2024-09-13T17:42:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.424+0000 7f2bc57fa640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 0x7f2bb4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.424+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3525+0+0 (secure 0 0 0) 0x7f2bc0046ef0 con 0x7f2bc809d690 2024-09-13T17:42:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.424+0000 7f2bc7fff640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 0x7f2bb4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:41.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.424+0000 7f2bc7fff640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 0x7f2bb4037910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80b1000 tx=0x7f2bb80aa000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:41.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.428+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bc00192c0 con 0x7f2bc809d690 2024-09-13T17:42:41.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:41.616+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2bc8138050 con 0x7f2bc809d690 2024-09-13T17:42:42.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.292+0000 7f2bc57fa640 1 -- 172.21.15.96:0/2802881070 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2bc8138050 con 0x7f2bc809d690 2024-09-13T17:42:42.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 msgr2=0x7f2bb4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 0x7f2bb4037910 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2bb80b1000 tx=0x7f2bb80aa000 comp rx=0 tx=0).stop 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 msgr2=0x7f2bc81c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c3b0 tx=0x7f2bc000c870 comp rx=0 tx=0).stop 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 shutdown_connections 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2bb4035450 0x7f2bb4037910 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 --2- 172.21.15.96:0/2802881070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc809d690 0x7f2bc81c94e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 >> 172.21.15.96:0/2802881070 conn(0x7f2bc806bc60 msgr2=0x7f2bc8131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 shutdown_connections 2024-09-13T17:42:42.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.296+0000 7f2bcee92640 1 -- 172.21.15.96:0/2802881070 wait complete. 2024-09-13T17:42:42.321 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:42:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfea737640 1 Processor -- start 2024-09-13T17:42:42.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfea737640 1 -- start start 2024-09-13T17:42:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfea737640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfe409e890 0x7fcfe413ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfea737640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe409e0d0 con 0x7fcfe4135e80 2024-09-13T17:42:42.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfea737640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe409ef50 con 0x7fcfe409e890 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfe409e890 0x7fcfe413ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcfe409e890 0x7fcfe413ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55496/0 (socket says 172.21.15.96:55496) 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcfe4135e80 0x7fcfe4136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:35988/0 (socket says 172.21.15.96:35988) 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe37fe640 1 -- 172.21.15.96:0/2582711219 learned_addr learned my addr 172.21.15.96:0/2582711219 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe37fe640 1 -- 172.21.15.96:0/2582711219 >> v1:172.21.15.96:6789/0 conn(0x7fcfe4135e80 legacy=0x7fcfe4136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe37fe640 1 -- 172.21.15.96:0/2582711219 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe409bf40 con 0x7fcfe409e890 2024-09-13T17:42:42.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.488+0000 7fcfe2ffd640 1 -- 172.21.15.96:0/2582711219 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643723980 0 0) 0x7fcfe409e0d0 con 0x7fcfe4135e80 2024-09-13T17:42:42.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe37fe640 1 --2- 172.21.15.96:0/2582711219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe413ade0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097f0 tx=0x7fcfd801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc9fa91e39c88f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:42.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe2ffd640 1 -- 172.21.15.96:0/2582711219 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd80aa590 con 0x7fcfe409e890 2024-09-13T17:42:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe2ffd640 1 -- 172.21.15.96:0/2582711219 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80aa6f0 con 0x7fcfe409e890 2024-09-13T17:42:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe2ffd640 1 -- 172.21.15.96:0/2582711219 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd80aa9b0 con 0x7fcfe409e890 2024-09-13T17:42:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2582711219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 msgr2=0x7fcfe413ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:42.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 --2- 172.21.15.96:0/2582711219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe413ade0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80097f0 tx=0x7fcfd801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2582711219 shutdown_connections 2024-09-13T17:42:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 --2- 172.21.15.96:0/2582711219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe413ade0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2582711219 >> 172.21.15.96:0/2582711219 conn(0x7fcfe4098a60 msgr2=0x7fcfe409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2582711219 shutdown_connections 2024-09-13T17:42:42.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2582711219 wait complete. 2024-09-13T17:42:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 Processor -- start 2024-09-13T17:42:42.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- start start 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe406ff00 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55508/0 (socket says 172.21.15.96:55508) 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe3fff640 1 -- 172.21.15.96:0/2393395232 learned_addr learned my addr 172.21.15.96:0/2393395232 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe3fff640 1 -- 172.21.15.96:0/2393395232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80094d0 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe3fff640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c390 tx=0x7fcfd400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd4007410 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd4007590 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe411d5d0 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcfd40047f0 con 0x7fcfe409e890 2024-09-13T17:42:42.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.492+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe413aa60 con 0x7fcfe409e890 2024-09-13T17:42:42.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcfd4004950 con 0x7fcfe409e890 2024-09-13T17:42:42.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe413a510 con 0x7fcfe409e890 2024-09-13T17:42:42.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfe0ff9640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 0x7fcfd0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:42.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3525+0+0 (secure 0 0 0) 0x7fcfd4047460 con 0x7fcfe409e890 2024-09-13T17:42:42.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfe37fe640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 0x7fcfd0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:42.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.496+0000 7fcfe37fe640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 0x7fcfd0037910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80ae000 tx=0x7fcfd8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:42.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.500+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcfd401bb70 con 0x7fcfe409e890 2024-09-13T17:42:42.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:42.656+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcfe409d540 con 0x7fcfe409e890 2024-09-13T17:42:43.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.300+0000 7fcfe0ff9640 1 -- 172.21.15.96:0/2393395232 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v17) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcfd401b340 con 0x7fcfe409e890 2024-09-13T17:42:43.301 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:42:43.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.300+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 msgr2=0x7fcfd0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:43.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.300+0000 7fcfea737640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 0x7fcfd0037910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80ae000 tx=0x7fcfd8008040 comp rx=0 tx=0).stop 2024-09-13T17:42:43.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.300+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 msgr2=0x7fcfe40a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:43.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.300+0000 7fcfea737640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c390 tx=0x7fcfd400c850 comp rx=0 tx=0).stop 2024-09-13T17:42:43.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 shutdown_connections 2024-09-13T17:42:43.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcfd0035450 0x7fcfd0037910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:43.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 --2- 172.21.15.96:0/2393395232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfe409e890 0x7fcfe40a4a30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:43.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 >> 172.21.15.96:0/2393395232 conn(0x7fcfe4098a60 msgr2=0x7fcfe409a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:43.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 shutdown_connections 2024-09-13T17:42:43.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.304+0000 7fcfea737640 1 -- 172.21.15.96:0/2393395232 wait complete. 2024-09-13T17:42:43.328 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:42:43.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4fa55640 1 Processor -- start 2024-09-13T17:42:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4fa55640 1 -- start start 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4fa55640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a48135e80 0x7f0a4809e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4fa55640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4809e0d0 con 0x7f0a48136260 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4fa55640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a4809ef50 con 0x7f0a48135e80 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4d7f1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a48135e80 0x7f0a4809e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4d7f1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a48135e80 0x7f0a4809e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55522/0 (socket says 172.21.15.96:55522) 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4d7f1640 1 -- 172.21.15.96:0/2442700830 learned_addr learned my addr 172.21.15.96:0/2442700830 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4d7f1640 1 -- 172.21.15.96:0/2442700830 >> v1:172.21.15.96:6789/0 conn(0x7f0a48136260 legacy=0x7f0a4806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.452+0000 7f0a4d7f1640 1 -- 172.21.15.96:0/2442700830 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4809bf40 con 0x7f0a48135e80 2024-09-13T17:42:43.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 --2- 172.21.15.96:0/2442700830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a4809e890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009e50 tx=0x7f0a4001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b44b7790bc25ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:43.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3ffff640 1 -- 172.21.15.96:0/2442700830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a40009200 con 0x7f0a48135e80 2024-09-13T17:42:43.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3ffff640 1 -- 172.21.15.96:0/2442700830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a400a5790 con 0x7f0a48135e80 2024-09-13T17:42:43.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3ffff640 1 -- 172.21.15.96:0/2442700830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a400a5a50 con 0x7f0a48135e80 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2442700830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 msgr2=0x7f0a4809e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2442700830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a4809e890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009e50 tx=0x7f0a4001ca30 comp rx=0 tx=0).stop 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2442700830 shutdown_connections 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2442700830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a4809e890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2442700830 >> 172.21.15.96:0/2442700830 conn(0x7f0a48098a60 msgr2=0x7f0a4809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2442700830 shutdown_connections 2024-09-13T17:42:43.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2442700830 wait complete. 2024-09-13T17:42:43.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 Processor -- start 2024-09-13T17:42:43.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- start start 2024-09-13T17:42:43.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:43.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480a3bc0 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55534/0 (socket says 172.21.15.96:55534) 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 -- 172.21.15.96:0/2457995084 learned_addr learned my addr 172.21.15.96:0/2457995084 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 -- 172.21.15.96:0/2457995084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a40009b30 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4d7f1640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0a4001a660 tx=0x7f0a400197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a400b87b0 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a400b8910 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a480a3ed0 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a400b8bd0 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.456+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a480a2160 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0a400b8d30 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a480a3d00 con 0x7f0a48135e80 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a3dffb640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 0x7f0a2c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:43.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3087+0+0 (secure 0 0 0) 0x7f0a400e2ea0 con 0x7f0a48135e80 2024-09-13T17:42:43.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a4cff0640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 0x7f0a2c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:43.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.460+0000 7f0a4cff0640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 0x7f0a2c037910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000f70 tx=0x7f0a38007a40 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.464+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a400b5d70 con 0x7f0a48135e80 2024-09-13T17:42:43.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:43.620+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a480a2160 con 0x7f0a48135e80 2024-09-13T17:42:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.300+0000 7f0a3dffb640 1 -- 172.21.15.96:0/2457995084 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v18) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0a480a2160 con 0x7f0a48135e80 2024-09-13T17:42:44.303 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-13T17:42:44.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.308+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 msgr2=0x7f0a2c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:44.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.308+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 0x7f0a2c037910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000f70 tx=0x7f0a38007a40 comp rx=0 tx=0).stop 2024-09-13T17:42:44.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.308+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 msgr2=0x7f0a481ca4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:44.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.308+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0a4001a660 tx=0x7f0a400197d0 comp rx=0 tx=0).stop 2024-09-13T17:42:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 shutdown_connections 2024-09-13T17:42:44.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a2c035450 0x7f0a2c037910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 --2- 172.21.15.96:0/2457995084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a48135e80 0x7f0a481ca4b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 >> 172.21.15.96:0/2457995084 conn(0x7f0a48098a60 msgr2=0x7f0a4809a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 shutdown_connections 2024-09-13T17:42:44.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.312+0000 7f0a4fa55640 1 -- 172.21.15.96:0/2457995084 wait complete. 2024-09-13T17:42:44.337 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:42:44.338 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:42:44.338 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:42:44.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:42:44.338 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:42:44.338 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T17:42:44.338 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:42:44.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:42:44.351 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:42:44.351 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:42:44.351 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:42:44.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:42:44.398 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:42:44.398 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:44.399 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:42:44.399 DEBUG:teuthology.parallel:result is None 2024-09-13T17:42:44.399 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:42:44.460 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:42:44.462 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.512+0000 7f019fe06640 1 Processor -- start 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.512+0000 7f019fe06640 1 -- start start 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.512+0000 7f019fe06640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f019809ac70 0x7f019809e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.512+0000 7f019fe06640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019809f0c0 con 0x7f019806ac90 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.512+0000 7f019fe06640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f019809cde0 con 0x7f019809ac70 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019dba2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f019809ac70 0x7f019809e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019d3a1640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f019806ac90 0x7f019809b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36022/0 (socket says 172.21.15.96:36022) 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019dba2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f019809ac70 0x7f019809e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55586/0 (socket says 172.21.15.96:55586) 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019d3a1640 1 -- 172.21.15.96:0/421171567 learned_addr learned my addr 172.21.15.96:0/421171567 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:44.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019dba2640 1 -- 172.21.15.96:0/421171567 >> v1:172.21.15.96:6789/0 conn(0x7f019806ac90 legacy=0x7f019809b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:44.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019dba2640 1 -- 172.21.15.96:0/421171567 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019809be10 con 0x7f019809ac70 2024-09-13T17:42:44.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019cba0640 1 -- 172.21.15.96:0/421171567 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248974743 0 0) 0x7f019809f0c0 con 0x7f019806ac90 2024-09-13T17:42:44.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019dba2640 1 --2- 172.21.15.96:0/421171567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019809ac70 0x7f019809e970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0188009740 tx=0x7f018801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74811a4cbdc8c102 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019cba0640 1 -- 172.21.15.96:0/421171567 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01880ab530 con 0x7f019809ac70 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019cba0640 1 -- 172.21.15.96:0/421171567 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a7050 con 0x7f019809ac70 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019cba0640 1 -- 172.21.15.96:0/421171567 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01880ab7e0 con 0x7f019809ac70 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 -- 172.21.15.96:0/421171567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019809ac70 msgr2=0x7f019809e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 --2- 172.21.15.96:0/421171567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019809ac70 0x7f019809e970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0188009740 tx=0x7f018801c390 comp rx=0 tx=0).stop 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 -- 172.21.15.96:0/421171567 shutdown_connections 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 --2- 172.21.15.96:0/421171567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019809ac70 0x7f019809e970 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 -- 172.21.15.96:0/421171567 >> 172.21.15.96:0/421171567 conn(0x7f019806e350 msgr2=0x7f0198070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 -- 172.21.15.96:0/421171567 shutdown_connections 2024-09-13T17:42:44.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 -- 172.21.15.96:0/421171567 wait complete. 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.516+0000 7f019fe06640 1 Processor -- start 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 -- start start 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0198131f90 con 0x7f019806ac90 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019dba2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019dba2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55600/0 (socket says 172.21.15.96:55600) 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019dba2640 1 -- 172.21.15.96:0/2680794456 learned_addr learned my addr 172.21.15.96:0/2680794456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019dba2640 1 -- 172.21.15.96:0/2680794456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188009420 con 0x7f019806ac90 2024-09-13T17:42:44.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019dba2640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0188019fc0 tx=0x7f018801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:44.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01880b7450 con 0x7f019806ac90 2024-09-13T17:42:44.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880b75b0 con 0x7f019806ac90 2024-09-13T17:42:44.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01980a4380 con 0x7f019806ac90 2024-09-13T17:42:44.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f01880b7850 con 0x7f019806ac90 2024-09-13T17:42:44.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01981354a0 con 0x7f019806ac90 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019809e970 con 0x7f019806ac90 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f01880b79b0 con 0x7f019806ac90 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 0x7f017c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 2673+0+0 (secure 0 0 0) 0x7f01880e1e80 con 0x7f019806ac90 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019d3a1640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 0x7f017c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:44.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.520+0000 7f019d3a1640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 0x7f017c037910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0194007fd0 tx=0x7f0194007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:44.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.524+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f01880c7950 con 0x7f019806ac90 2024-09-13T17:42:44.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:44.700+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f019806b3e0 con 0x7f019806ac90 2024-09-13T17:42:45.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.492+0000 7f018e7fc640 1 -- 172.21.15.96:0/2680794456 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v21) v1 ==== 97+0+0 (secure 0 0 0) 0x7f01880b0030 con 0x7f019806ac90 2024-09-13T17:42:45.496 INFO:teuthology.orchestra.run.smithi096.stderr: removed all blocklist entries 2024-09-13T17:42:45.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 msgr2=0x7f017c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 0x7f017c037910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0194007fd0 tx=0x7f0194007920 comp rx=0 tx=0).stop 2024-09-13T17:42:45.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 msgr2=0x7f01981302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0188019fc0 tx=0x7f018801abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 shutdown_connections 2024-09-13T17:42:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f017c035450 0x7f017c037910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 --2- 172.21.15.96:0/2680794456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f019806ac90 0x7f01981302f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 >> 172.21.15.96:0/2680794456 conn(0x7f019806e350 msgr2=0x7f019806fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 shutdown_connections 2024-09-13T17:42:45.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.496+0000 7f019fe06640 1 -- 172.21.15.96:0/2680794456 wait complete. 2024-09-13T17:42:45.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:42:45.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f180143b640 1 Processor -- start 2024-09-13T17:42:45.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f180143b640 1 -- start start 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f180143b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f17fc134e70 0x7f17fc09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f180143b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17fc09ed60 con 0x7f17fc135250 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f180143b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17fc09ddc0 con 0x7f17fc134e70 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f17faffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f17fc134e70 0x7f17fc09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f17faffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f17fc134e70 0x7f17fc09e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55616/0 (socket says 172.21.15.96:55616) 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f17faffd640 1 -- 172.21.15.96:0/366245564 learned_addr learned my addr 172.21.15.96:0/366245564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.644+0000 7f17fa7fc640 1 --1- 172.21.15.96:0/366245564 >> v1:172.21.15.96:6789/0 conn(0x7f17fc135250 0x7f17fc135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36030/0 (socket says 172.21.15.96:36030) 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17faffd640 1 -- 172.21.15.96:0/366245564 >> v1:172.21.15.96:6789/0 conn(0x7f17fc135250 legacy=0x7f17fc135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17faffd640 1 -- 172.21.15.96:0/366245564 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17fc0a68b0 con 0x7f17fc134e70 2024-09-13T17:42:45.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17faffd640 1 --2- 172.21.15.96:0/366245564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc09e820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f17e40097f0 tx=0x7f17e401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49abcdbcb5f24027 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:45.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17f9ffb640 1 -- 172.21.15.96:0/366245564 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e40ac540 con 0x7f17fc134e70 2024-09-13T17:42:45.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17f9ffb640 1 -- 172.21.15.96:0/366245564 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e40ac6a0 con 0x7f17fc134e70 2024-09-13T17:42:45.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f17f9ffb640 1 -- 172.21.15.96:0/366245564 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e40ac960 con 0x7f17fc134e70 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 -- 172.21.15.96:0/366245564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 msgr2=0x7f17fc09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 --2- 172.21.15.96:0/366245564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc09e820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f17e40097f0 tx=0x7f17e401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 -- 172.21.15.96:0/366245564 shutdown_connections 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 --2- 172.21.15.96:0/366245564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc09e820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 -- 172.21.15.96:0/366245564 >> 172.21.15.96:0/366245564 conn(0x7f17fc098a60 msgr2=0x7f17fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 -- 172.21.15.96:0/366245564 shutdown_connections 2024-09-13T17:42:45.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.648+0000 7f180143b640 1 -- 172.21.15.96:0/366245564 wait complete. 2024-09-13T17:42:45.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 Processor -- start 2024-09-13T17:42:45.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 -- start start 2024-09-13T17:42:45.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:45.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17fc0a1710 con 0x7f17fc134e70 2024-09-13T17:42:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55626/0 (socket says 172.21.15.96:55626) 2024-09-13T17:42:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17faffd640 1 -- 172.21.15.96:0/3410791935 learned_addr learned my addr 172.21.15.96:0/3410791935 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17faffd640 1 -- 172.21.15.96:0/3410791935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e40094d0 con 0x7f17fc134e70 2024-09-13T17:42:45.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17faffd640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f17e40190f0 tx=0x7f17e4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:45.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e40bb070 con 0x7f17fc134e70 2024-09-13T17:42:45.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e40b7470 con 0x7f17fc134e70 2024-09-13T17:42:45.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17fc0a3550 con 0x7f17fc134e70 2024-09-13T17:42:45.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f17e40b7730 con 0x7f17fc134e70 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17fc0a4270 con 0x7f17fc134e70 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17fc0a3350 con 0x7f17fc134e70 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f17e40b3070 con 0x7f17fc134e70 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 0x7f17dc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f17e40e1450 con 0x7f17fc134e70 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17fa7fc640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 0x7f17dc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:45.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.652+0000 7f17fa7fc640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 0x7f17dc037910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17ec001d00 tx=0x7f17ec007dd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:45.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.656+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f17e40b4380 con 0x7f17fc134e70 2024-09-13T17:42:45.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.868+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f17fc1368b0 con 0x7f17fc134e70 2024-09-13T17:42:45.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f17f37fe640 1 -- 172.21.15.96:0/3410791935 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+3875 (secure 0 0 0) 0x7f17e40b1070 con 0x7f17fc134e70 2024-09-13T17:42:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:45.873 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:42:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_dump": [ 2024-09-13T17:42:45.873 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.a", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmAVcqLxAAIRo/psndnU3y/VG+/goNRQ==", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.b", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmpF6ANxAAzrn1jVVSCWKv5q1kdChbWA==", 2024-09-13T17:42:45.874 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.0", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBKeeRmkXcoFhAAdX1HBKKYCHqtacYyAZbF2w==", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:42:45.875 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.1", 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBNeeRmh189GxAAFDpo1k0lxmmillsXoiKXMA==", 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.876 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.2", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBQeeRmKpvLIhAA4Y7okMAZAQ9cjYpkaKF/kA==", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.0", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRm3enLAxAAZGIwl+Bs06GY4JG0dbxLBQ==", 2024-09-13T17:42:45.877 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.1", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRmNlIQCRAAFFcBZ0Pwtq8kXtBAAFkuDA==", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-13T17:42:45.878 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.admin", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmUwoTGBAAycvCNfOc1ueY+QjBkpAqpA==", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-mds", 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmAgFmBRAAlaXYcEofvEGB57oxS8Fiog==", 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-mds" 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-mgr", 2024-09-13T17:42:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmZwpmBRAAB3a3ObJY/5hztk5ggBdbjw==", 2024-09-13T17:42:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.881 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-osd", 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmuhJmBRAANlQlxZAvyWW5Uy3XrcAhyQ==", 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-osd" 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rbd", 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmxxpmBRAAZN2W9LZ1OaNk2/6d28DbmA==", 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmSiRmBRAAIdRrW5vETZV7PAbOx7OSBA==", 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-13T17:42:45.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.bootstrap-rgw", 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBVeeRmiCxmBRAA7VchgkgwUb2aWYBro5LX3w==", 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.x", 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmOOcEKhAAJmefIyHpVdx4zDzhwQcawA==", 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:42:45.884 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:42:45.885 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 msgr2=0x7f17dc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 0x7f17dc037910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f17ec001d00 tx=0x7f17ec007dd0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 msgr2=0x7f17fc12fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f17e40190f0 tx=0x7f17e4019280 comp rx=0 tx=0).stop 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 shutdown_connections 2024-09-13T17:42:45.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f17dc035450 0x7f17dc037910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 --2- 172.21.15.96:0/3410791935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17fc134e70 0x7f17fc12fa60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 >> 172.21.15.96:0/3410791935 conn(0x7f17fc098a60 msgr2=0x7f17fc133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 shutdown_connections 2024-09-13T17:42:45.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:45.872+0000 7f180143b640 1 -- 172.21.15.96:0/3410791935 wait complete. 2024-09-13T17:42:45.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-13T17:42:46.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab41b0640 1 Processor -- start 2024-09-13T17:42:46.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab41b0640 1 -- start start 2024-09-13T17:42:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab41b0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdaac06ff00 0x7fdaac141340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab41b0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac0710a0 con 0x7fdaac13c4d0 2024-09-13T17:42:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab41b0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaac13c160 con 0x7fdaac06ff00 2024-09-13T17:42:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab1f4c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdaac13c4d0 0x7fdaac13c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36038/0 (socket says 172.21.15.96:36038) 2024-09-13T17:42:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab1f4c640 1 -- 172.21.15.96:0/4270965841 learned_addr learned my addr 172.21.15.96:0/4270965841 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:46.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab174b640 1 --2- 172.21.15.96:0/4270965841 >> v2:172.21.15.96:3300/0 conn(0x7fdaac06ff00 0x7fdaac141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab174b640 1 -- 172.21.15.96:0/4270965841 >> v1:172.21.15.96:6789/0 conn(0x7fdaac13c4d0 legacy=0x7fdaac13c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.008+0000 7fdab174b640 1 -- 172.21.15.96:0/4270965841 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac071bd0 con 0x7fdaac06ff00 2024-09-13T17:42:46.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab174b640 1 --2- 172.21.15.96:0/4270965841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac141340 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009d90 tx=0x7fdaa001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0045d787d1c2b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab0f4a640 1 -- 172.21.15.96:0/4270965841 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdaa00a57b0 con 0x7fdaac06ff00 2024-09-13T17:42:46.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab0f4a640 1 -- 172.21.15.96:0/4270965841 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa00a5910 con 0x7fdaac06ff00 2024-09-13T17:42:46.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab0f4a640 1 -- 172.21.15.96:0/4270965841 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdaa00a5bd0 con 0x7fdaac06ff00 2024-09-13T17:42:46.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/4270965841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 msgr2=0x7fdaac141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 --2- 172.21.15.96:0/4270965841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac141340 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009d90 tx=0x7fdaa001c880 comp rx=0 tx=0).stop 2024-09-13T17:42:46.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/4270965841 shutdown_connections 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 --2- 172.21.15.96:0/4270965841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac141340 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/4270965841 >> 172.21.15.96:0/4270965841 conn(0x7fdaac098a60 msgr2=0x7fdaac09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/4270965841 shutdown_connections 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/4270965841 wait complete. 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 Processor -- start 2024-09-13T17:42:46.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- start start 2024-09-13T17:42:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac140b50 con 0x7fdaac06ff00 2024-09-13T17:42:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab1f4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab1f4c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55646/0 (socket says 172.21.15.96:55646) 2024-09-13T17:42:46.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab1f4c640 1 -- 172.21.15.96:0/227569031 learned_addr learned my addr 172.21.15.96:0/227569031 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab1f4c640 1 -- 172.21.15.96:0/227569031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa0009a70 con 0x7fdaac06ff00 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab1f4c640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c3a0 tx=0x7fdaa400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdaa4005c20 con 0x7fdaac06ff00 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa4005d80 con 0x7fdaac06ff00 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdaa4007200 con 0x7fdaac06ff00 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac09d320 con 0x7fdaac06ff00 2024-09-13T17:42:46.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.012+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac13bb70 con 0x7fdaac06ff00 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fdaa4007360 con 0x7fdaac06ff00 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac09d060 con 0x7fdaac06ff00 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdaaa7fc640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 0x7fda980378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7fdaa4046870 con 0x7fdaac06ff00 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdab174b640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 0x7fda980378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.016+0000 7fdab174b640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 0x7fda980378c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00af000 tx=0x7fdaa00a7070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.020+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdaa401b3b0 con 0x7fdaac06ff00 2024-09-13T17:42:46.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.232+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fdaac09d7b0 con 0x7fdaac06ff00 2024-09-13T17:42:46.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdaaa7fc640 1 -- 172.21.15.96:0/227569031 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdaa4017030 con 0x7fdaac06ff00 2024-09-13T17:42:46.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 msgr2=0x7fda980378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 0x7fda980378c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00af000 tx=0x7fdaa00a7070 comp rx=0 tx=0).stop 2024-09-13T17:42:46.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 msgr2=0x7fdaac0a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdaa400c3a0 tx=0x7fdaa400c860 comp rx=0 tx=0).stop 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 shutdown_connections 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fda98035400 0x7fda980378c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 --2- 172.21.15.96:0/227569031 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaac06ff00 0x7fdaac0a66b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 >> 172.21.15.96:0/227569031 conn(0x7fdaac098a60 msgr2=0x7fdaac1359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 shutdown_connections 2024-09-13T17:42:46.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.236+0000 7fdab41b0640 1 -- 172.21.15.96:0/227569031 wait complete. 2024-09-13T17:42:46.254 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-13T17:42:46.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39d13f9640 1 Processor -- start 2024-09-13T17:42:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39d13f9640 1 -- start start 2024-09-13T17:42:46.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39d13f9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f39cc13af50 0x7f39cc13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39d13f9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc138050 con 0x7f39cc09d690 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39d13f9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc09c410 con 0x7f39cc13af50 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39ca7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f39cc13af50 0x7f39cc13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39ca7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f39cc13af50 0x7f39cc13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55648/0 (socket says 172.21.15.96:55648) 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39caffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f39cc09d690 0x7f39cc138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36044/0 (socket says 172.21.15.96:36044) 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39ca7fc640 1 -- 172.21.15.96:0/2781451507 learned_addr learned my addr 172.21.15.96:0/2781451507 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39ca7fc640 1 -- 172.21.15.96:0/2781451507 >> v1:172.21.15.96:6789/0 conn(0x7f39cc09d690 legacy=0x7f39cc138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.360+0000 7f39ca7fc640 1 -- 172.21.15.96:0/2781451507 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc09cba0 con 0x7f39cc13af50 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39ca7fc640 1 --2- 172.21.15.96:0/2781451507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc13af50 0x7f39cc13d340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097f0 tx=0x7f39b401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee23733afa5ff8d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c9ffb640 1 -- 172.21.15.96:0/2781451507 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39b40a8900 con 0x7f39cc13af50 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c9ffb640 1 -- 172.21.15.96:0/2781451507 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b40a8a60 con 0x7f39cc13af50 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c9ffb640 1 -- 172.21.15.96:0/2781451507 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39b40a8d20 con 0x7f39cc13af50 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/2781451507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc13af50 msgr2=0x7f39cc13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 --2- 172.21.15.96:0/2781451507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc13af50 0x7f39cc13d340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f39b40097f0 tx=0x7f39b401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/2781451507 shutdown_connections 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 --2- 172.21.15.96:0/2781451507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc13af50 0x7f39cc13d340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/2781451507 >> 172.21.15.96:0/2781451507 conn(0x7f39cc06bc60 msgr2=0x7f39cc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/2781451507 shutdown_connections 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/2781451507 wait complete. 2024-09-13T17:42:46.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 Processor -- start 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- start start 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc0a2be0 con 0x7f39cc09d690 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55656/0 (socket says 172.21.15.96:55656) 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39caffd640 1 -- 172.21.15.96:0/3827923716 learned_addr learned my addr 172.21.15.96:0/3827923716 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39caffd640 1 -- 172.21.15.96:0/3827923716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40094d0 con 0x7f39cc09d690 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39caffd640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c700 tx=0x7f39bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39bc005db0 con 0x7f39cc09d690 2024-09-13T17:42:46.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc00a660 con 0x7f39cc09d690 2024-09-13T17:42:46.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f39bc00a900 con 0x7f39cc09d690 2024-09-13T17:42:46.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc0a51a0 con 0x7f39cc09d690 2024-09-13T17:42:46.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc0a35d0 con 0x7f39cc09d690 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f39bc00aa60 con 0x7f39cc09d690 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc0a33b0 con 0x7f39cc09d690 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.364+0000 7f39c37fe640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 0x7f39ac037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.368+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f39bc046960 con 0x7f39cc09d690 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.368+0000 7f39ca7fc640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 0x7f39ac037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.368+0000 7f39ca7fc640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 0x7f39ac037910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b2000 tx=0x7f39b40aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.372+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f39bc00ad10 con 0x7f39cc09d690 2024-09-13T17:42:46.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.580+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f39cc138050 con 0x7f39cc09d690 2024-09-13T17:42:46.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.584+0000 7f39c37fe640 1 -- 172.21.15.96:0/3827923716 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f39cc138050 con 0x7f39cc09d690 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 msgr2=0x7f39ac037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 0x7f39ac037910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f39b40b2000 tx=0x7f39b40aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 msgr2=0x7f39cc1c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c700 tx=0x7f39bc00cbc0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 shutdown_connections 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f39ac035450 0x7f39ac037910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 --2- 172.21.15.96:0/3827923716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39cc09d690 0x7f39cc1c94e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 >> 172.21.15.96:0/3827923716 conn(0x7f39cc06bc60 msgr2=0x7f39cc131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 shutdown_connections 2024-09-13T17:42:46.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.588+0000 7f39d13f9640 1 -- 172.21.15.96:0/3827923716 wait complete. 2024-09-13T17:42:46.606 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff2833c3640 1 Processor -- start 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff2833c3640 1 -- start start 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff2833c3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff27c134e70 0x7ff27c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff2833c3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c09ed60 con 0x7ff27c135250 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff2833c3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c09ddc0 con 0x7ff27c134e70 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff27c134e70 0x7ff27c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff27c134e70 0x7ff27c09e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55668/0 (socket says 172.21.15.96:55668) 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 -- 172.21.15.96:0/3281940283 learned_addr learned my addr 172.21.15.96:0/3281940283 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28095e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff27c135250 0x7ff27c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36052/0 (socket says 172.21.15.96:36052) 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 -- 172.21.15.96:0/3281940283 >> v1:172.21.15.96:6789/0 conn(0x7ff27c135250 legacy=0x7ff27c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 -- 172.21.15.96:0/3281940283 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0a68b0 con 0x7ff27c134e70 2024-09-13T17:42:46.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff28115f640 1 --2- 172.21.15.96:0/3281940283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09e820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2780097f0 tx=0x7ff27801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2845f79871ed4831 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff273fff640 1 -- 172.21.15.96:0/3281940283 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2780ac540 con 0x7ff27c134e70 2024-09-13T17:42:46.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff273fff640 1 -- 172.21.15.96:0/3281940283 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780ac6a0 con 0x7ff27c134e70 2024-09-13T17:42:46.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.724+0000 7ff273fff640 1 -- 172.21.15.96:0/3281940283 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2780ac960 con 0x7ff27c134e70 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/3281940283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 msgr2=0x7ff27c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 --2- 172.21.15.96:0/3281940283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09e820 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff2780097f0 tx=0x7ff27801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/3281940283 shutdown_connections 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 --2- 172.21.15.96:0/3281940283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09e820 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/3281940283 >> 172.21.15.96:0/3281940283 conn(0x7ff27c098a60 msgr2=0x7ff27c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/3281940283 shutdown_connections 2024-09-13T17:42:46.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/3281940283 wait complete. 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 Processor -- start 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- start start 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0a1dc0 con 0x7ff27c134e70 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28115f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28115f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55678/0 (socket says 172.21.15.96:55678) 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28115f640 1 -- 172.21.15.96:0/2015740355 learned_addr learned my addr 172.21.15.96:0/2015740355 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28115f640 1 -- 172.21.15.96:0/2015740355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2780094d0 con 0x7ff27c134e70 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28115f640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff2780190f0 tx=0x7ff278019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2780bb070 con 0x7ff27c134e70 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2780b7470 con 0x7ff27c134e70 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c0a2a60 con 0x7ff27c134e70 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2780b7730 con 0x7ff27c134e70 2024-09-13T17:42:46.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c13ba70 con 0x7ff27c134e70 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c12eed0 con 0x7ff27c134e70 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff2780b3070 con 0x7ff27c134e70 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 0x7ff260037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7ff2780e1450 con 0x7ff27c134e70 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28095e640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 0x7ff260037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:46.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.728+0000 7ff28095e640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 0x7ff260037910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001600 tx=0x7ff26c007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:46.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.732+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff2780b4380 con 0x7ff27c134e70 2024-09-13T17:42:46.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.943+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7ff27c1368b0 con 0x7ff27c134e70 2024-09-13T17:42:46.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.947+0000 7ff271ffb640 1 -- 172.21.15.96:0/2015740355 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff2780b1070 con 0x7ff27c134e70 2024-09-13T17:42:46.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 msgr2=0x7ff260037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 0x7ff260037910 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001600 tx=0x7ff26c007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:46.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 msgr2=0x7ff27c09f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff2780190f0 tx=0x7ff278019280 comp rx=0 tx=0).stop 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 shutdown_connections 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff260035450 0x7ff260037910 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 --2- 172.21.15.96:0/2015740355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff27c134e70 0x7ff27c09f280 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 >> 172.21.15.96:0/2015740355 conn(0x7ff27c098a60 msgr2=0x7ff27c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 shutdown_connections 2024-09-13T17:42:46.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:46.951+0000 7ff2833c3640 1 -- 172.21.15.96:0/2015740355 wait complete. 2024-09-13T17:42:46.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 Processor -- start 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- start start 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e28134e70 0x7f8e2809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2809ed60 con 0x7f8e28135250 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e2809ddc0 con 0x7f8e28134e70 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e26d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e28134e70 0x7f8e2809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e1ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8e28135250 0x7f8e28135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36056/0 (socket says 172.21.15.96:36056) 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e26d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e28134e70 0x7f8e2809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55680/0 (socket says 172.21.15.96:55680) 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e1ffff640 1 -- 172.21.15.96:0/3780432733 learned_addr learned my addr 172.21.15.96:0/3780432733 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e26d9d640 1 -- 172.21.15.96:0/3780432733 >> v1:172.21.15.96:6789/0 conn(0x7f8e28135250 legacy=0x7f8e28135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e26d9d640 1 -- 172.21.15.96:0/3780432733 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280a68b0 con 0x7f8e28134e70 2024-09-13T17:42:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e26d9d640 1 --2- 172.21.15.96:0/3780432733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809e820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009740 tx=0x7f8e1801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f87b421ee9f70 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2659c640 1 -- 172.21.15.96:0/3780432733 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e180ab530 con 0x7f8e28134e70 2024-09-13T17:42:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2659c640 1 -- 172.21.15.96:0/3780432733 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180a7050 con 0x7f8e28134e70 2024-09-13T17:42:47.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2659c640 1 -- 172.21.15.96:0/3780432733 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e180ab7e0 con 0x7f8e28134e70 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- 172.21.15.96:0/3780432733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 msgr2=0x7f8e2809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 --2- 172.21.15.96:0/3780432733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809e820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8e18009740 tx=0x7f8e1801c420 comp rx=0 tx=0).stop 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- 172.21.15.96:0/3780432733 shutdown_connections 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 --2- 172.21.15.96:0/3780432733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809e820 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- 172.21.15.96:0/3780432733 >> 172.21.15.96:0/3780432733 conn(0x7f8e28098a60 msgr2=0x7f8e2809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- 172.21.15.96:0/3780432733 shutdown_connections 2024-09-13T17:42:47.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.075+0000 7f8e2d093640 1 -- 172.21.15.96:0/3780432733 wait complete. 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 Processor -- start 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 -- start start 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280a1dc0 con 0x7f8e28134e70 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e26d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e26d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55688/0 (socket says 172.21.15.96:55688) 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e26d9d640 1 -- 172.21.15.96:0/2074206720 learned_addr learned my addr 172.21.15.96:0/2074206720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e26d9d640 1 -- 172.21.15.96:0/2074206720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18009420 con 0x7f8e28134e70 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e26d9d640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e1801a050 tx=0x7f8e180191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e180abea0 con 0x7f8e28134e70 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e180b4410 con 0x7f8e28134e70 2024-09-13T17:42:47.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e280a2a60 con 0x7f8e28134e70 2024-09-13T17:42:47.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e180b46b0 con 0x7f8e28134e70 2024-09-13T17:42:47.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2813ba70 con 0x7f8e28134e70 2024-09-13T17:42:47.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e281368b0 con 0x7f8e28134e70 2024-09-13T17:42:47.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8e180b2040 con 0x7f8e28134e70 2024-09-13T17:42:47.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 0x7f8e08037960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f8e180e1470 con 0x7f8e28134e70 2024-09-13T17:42:47.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1ffff640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 0x7f8e08037960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.079+0000 7f8e1ffff640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 0x7f8e08037960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e10001610 tx=0x7f8e10007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.083+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e180bc3b0 con 0x7f8e28134e70 2024-09-13T17:42:47.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.295+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8e2812eac0 con 0x7f8e28134e70 2024-09-13T17:42:47.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.299+0000 7f8e1f7fe640 1 -- 172.21.15.96:0/2074206720 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8e180b0030 con 0x7f8e28134e70 2024-09-13T17:42:47.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 msgr2=0x7f8e08037960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 0x7f8e08037960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8e10001610 tx=0x7f8e10007ad0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 msgr2=0x7f8e2809f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8e1801a050 tx=0x7f8e180191c0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 shutdown_connections 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8e080354a0 0x7f8e08037960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 --2- 172.21.15.96:0/2074206720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e28134e70 0x7f8e2809f280 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 >> 172.21.15.96:0/2074206720 conn(0x7f8e28098a60 msgr2=0x7f8e28098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 shutdown_connections 2024-09-13T17:42:47.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.303+0000 7f8e2d093640 1 -- 172.21.15.96:0/2074206720 wait complete. 2024-09-13T17:42:47.320 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-13T17:42:47.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd5667fe640 1 Processor -- start 2024-09-13T17:42:47.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd5667fe640 1 -- start start 2024-09-13T17:42:47.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd5667fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd56009dc20 0x7fd5601351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd5667fe640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd56006e130 con 0x7fd56006f6d0 2024-09-13T17:42:47.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd5667fe640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5600a6750 con 0x7fd56009dc20 2024-09-13T17:42:47.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd56009dc20 0x7fd5601351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd56009dc20 0x7fd5601351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55690/0 (socket says 172.21.15.96:55690) 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 -- 172.21.15.96:0/38278408 learned_addr learned my addr 172.21.15.96:0/38278408 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55ffff640 1 --1- 172.21.15.96:0/38278408 >> v1:172.21.15.96:6789/0 conn(0x7fd56006f6d0 0x7fd560134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36060/0 (socket says 172.21.15.96:36060) 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 -- 172.21.15.96:0/38278408 >> v1:172.21.15.96:6789/0 conn(0x7fd56006f6d0 legacy=0x7fd560134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 -- 172.21.15.96:0/38278408 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5600712e0 con 0x7fd56009dc20 2024-09-13T17:42:47.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55effd640 1 -- 172.21.15.96:0/38278408 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912748803 0 0) 0x7fd56006e130 con 0x7fd56006f6d0 2024-09-13T17:42:47.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55f7fe640 1 --2- 172.21.15.96:0/38278408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56009dc20 0x7fd5601351c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5540097f0 tx=0x7fd55401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02a2f52826cadd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55effd640 1 -- 172.21.15.96:0/38278408 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5540ab400 con 0x7fd56009dc20 2024-09-13T17:42:47.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55effd640 1 -- 172.21.15.96:0/38278408 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540a7050 con 0x7fd56009dc20 2024-09-13T17:42:47.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.435+0000 7fd55effd640 1 -- 172.21.15.96:0/38278408 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5540ab6b0 con 0x7fd56009dc20 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/38278408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56009dc20 msgr2=0x7fd5601351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 --2- 172.21.15.96:0/38278408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56009dc20 0x7fd5601351c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd5540097f0 tx=0x7fd55401c450 comp rx=0 tx=0).stop 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/38278408 shutdown_connections 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 --2- 172.21.15.96:0/38278408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56009dc20 0x7fd5601351c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/38278408 >> 172.21.15.96:0/38278408 conn(0x7fd560098a60 msgr2=0x7fd56009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/38278408 shutdown_connections 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/38278408 wait complete. 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 Processor -- start 2024-09-13T17:42:47.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- start start 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56009f930 con 0x7fd56006f6d0 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55706/0 (socket says 172.21.15.96:55706) 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55ffff640 1 -- 172.21.15.96:0/2644579793 learned_addr learned my addr 172.21.15.96:0/2644579793 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55ffff640 1 -- 172.21.15.96:0/2644579793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540094d0 con 0x7fd56006f6d0 2024-09-13T17:42:47.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55ffff640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c420 tx=0x7fd55000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5500074d0 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550007630 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56006e660 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd550004960 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56006f400 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd550004ac0 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd560134ab0 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 0x7fd54c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7fd5500468c0 con 0x7fd56006f6d0 2024-09-13T17:42:47.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.439+0000 7fd55f7fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 0x7fd54c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.443+0000 7fd55f7fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 0x7fd54c037910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ad000 tx=0x7fd554008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.447+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd55001ab20 con 0x7fd56006f6d0 2024-09-13T17:42:47.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.647+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd56006a1b0 con 0x7fd56006f6d0 2024-09-13T17:42:47.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd55cff9640 1 -- 172.21.15.96:0/2644579793 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd550018020 con 0x7fd56006f6d0 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 msgr2=0x7fd54c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 0x7fd54c037910 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd5540ad000 tx=0x7fd554008040 comp rx=0 tx=0).stop 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 msgr2=0x7fd5600a4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fd55000c420 tx=0x7fd55000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 shutdown_connections 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd54c035450 0x7fd54c037910 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 --2- 172.21.15.96:0/2644579793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd56006f6d0 0x7fd5600a4d00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 >> 172.21.15.96:0/2644579793 conn(0x7fd560098a60 msgr2=0x7fd560099570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 shutdown_connections 2024-09-13T17:42:47.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.655+0000 7fd5667fe640 1 -- 172.21.15.96:0/2644579793 wait complete. 2024-09-13T17:42:47.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-13T17:42:47.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff0adb8640 1 Processor -- start 2024-09-13T17:42:47.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff0adb8640 1 -- start start 2024-09-13T17:42:47.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff0adb8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7eff0406f6d0 0x7eff04134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff0adb8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0406e130 con 0x7eff0409dc20 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff0adb8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040a6750 con 0x7eff0406f6d0 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff03fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7eff0409dc20 0x7eff04134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36066/0 (socket says 172.21.15.96:36066) 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff03fff640 1 -- 172.21.15.96:0/2342494482 learned_addr learned my addr 172.21.15.96:0/2342494482 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff08b54640 1 --2- 172.21.15.96:0/2342494482 >> v2:172.21.15.96:3300/0 conn(0x7eff0406f6d0 0x7eff04134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff037fe640 1 -- 172.21.15.96:0/2342494482 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790747446 0 0) 0x7eff0406e130 con 0x7eff0409dc20 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff037fe640 1 -- 172.21.15.96:0/2342494482 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efeec003530 con 0x7eff0409dc20 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff08b54640 1 -- 172.21.15.96:0/2342494482 >> v1:172.21.15.96:6789/0 conn(0x7eff0409dc20 legacy=0x7eff04134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff08b54640 1 -- 172.21.15.96:0/2342494482 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040712e0 con 0x7eff0406f6d0 2024-09-13T17:42:47.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.779+0000 7eff08b54640 1 --2- 172.21.15.96:0/2342494482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff04134ab0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efef4009740 tx=0x7efef401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96128276da21d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff037fe640 1 -- 172.21.15.96:0/2342494482 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef40ab400 con 0x7eff0406f6d0 2024-09-13T17:42:47.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff037fe640 1 -- 172.21.15.96:0/2342494482 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40a7050 con 0x7eff0406f6d0 2024-09-13T17:42:47.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff037fe640 1 -- 172.21.15.96:0/2342494482 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef40ab6b0 con 0x7eff0406f6d0 2024-09-13T17:42:47.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/2342494482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 msgr2=0x7eff04134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 --2- 172.21.15.96:0/2342494482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff04134ab0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7efef4009740 tx=0x7efef401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/2342494482 shutdown_connections 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 --2- 172.21.15.96:0/2342494482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff04134ab0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/2342494482 >> 172.21.15.96:0/2342494482 conn(0x7eff04098a60 msgr2=0x7eff0409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/2342494482 shutdown_connections 2024-09-13T17:42:47.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/2342494482 wait complete. 2024-09-13T17:42:47.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 Processor -- start 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- start start 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041207d0 con 0x7eff0406f6d0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff08b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff08b54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55712/0 (socket says 172.21.15.96:55712) 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff08b54640 1 -- 172.21.15.96:0/212643186 learned_addr learned my addr 172.21.15.96:0/212643186 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff08b54640 1 -- 172.21.15.96:0/212643186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4009420 con 0x7eff0406f6d0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff08b54640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efef4019fc0 tx=0x7efef401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef40abe10 con 0x7eff0406f6d0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40b7450 con 0x7eff0406f6d0 2024-09-13T17:42:47.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0411ec90 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efef40b76f0 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0411e940 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff041217e0 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7efef40b7850 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 0x7efef0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7efef40e1c90 con 0x7eff0406f6d0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.783+0000 7eff03fff640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 0x7efef0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:47.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.787+0000 7eff03fff640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 0x7efef0037910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efef8007fd0 tx=0x7efef8007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:47.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.791+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7eff041217e0 con 0x7eff0406f6d0 2024-09-13T17:42:47.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.991+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7eff041217e0 con 0x7eff0406f6d0 2024-09-13T17:42:48.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.995+0000 7eff017fa640 1 -- 172.21.15.96:0/212643186 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7eff041217e0 con 0x7eff0406f6d0 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 msgr2=0x7efef0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 0x7efef0037910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efef8007fd0 tx=0x7efef8007980 comp rx=0 tx=0).stop 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 msgr2=0x7eff0411cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7efef4019fc0 tx=0x7efef401abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 shutdown_connections 2024-09-13T17:42:48.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efef0035450 0x7efef0037910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 --2- 172.21.15.96:0/212643186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff0406f6d0 0x7eff0411cd30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 >> 172.21.15.96:0/212643186 conn(0x7eff04098a60 msgr2=0x7eff041393b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 shutdown_connections 2024-09-13T17:42:48.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:47.999+0000 7eff0adb8640 1 -- 172.21.15.96:0/212643186 wait complete. 2024-09-13T17:42:48.018 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:48.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.131+0000 7f8fcc1a2640 1 Processor -- start 2024-09-13T17:42:48.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.131+0000 7f8fcc1a2640 1 -- start start 2024-09-13T17:42:48.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fcc1a2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8fc4134e70 0x7f8fc409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fcc1a2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc409ed60 con 0x7f8fc4135250 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fcc1a2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc409ddc0 con 0x7f8fc4134e70 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc9f3e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8fc4134e70 0x7f8fc409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc973d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8fc4135250 0x7f8fc4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36074/0 (socket says 172.21.15.96:36074) 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc973d640 1 -- 172.21.15.96:0/1656032782 learned_addr learned my addr 172.21.15.96:0/1656032782 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc9f3e640 1 -- 172.21.15.96:0/1656032782 >> v1:172.21.15.96:6789/0 conn(0x7f8fc4135250 legacy=0x7f8fc4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc8f3c640 1 -- 172.21.15.96:0/1656032782 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633050560 0 0) 0x7f8fbc0014a0 con 0x7f8fc4135250 2024-09-13T17:42:48.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc9f3e640 1 -- 172.21.15.96:0/1656032782 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc40a68b0 con 0x7f8fc4134e70 2024-09-13T17:42:48.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc9f3e640 1 --2- 172.21.15.96:0/1656032782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc409e820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009740 tx=0x7f8fc001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819c9a001a14d47e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc8f3c640 1 -- 172.21.15.96:0/1656032782 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc00ab530 con 0x7f8fc4134e70 2024-09-13T17:42:48.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc8f3c640 1 -- 172.21.15.96:0/1656032782 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00a7050 con 0x7f8fc4134e70 2024-09-13T17:42:48.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fc8f3c640 1 -- 172.21.15.96:0/1656032782 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc00ab7e0 con 0x7f8fc4134e70 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/1656032782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 msgr2=0x7f8fc409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.135+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/1656032782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc409e820 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009740 tx=0x7f8fc001c420 comp rx=0 tx=0).stop 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/1656032782 shutdown_connections 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/1656032782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc409e820 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/1656032782 >> 172.21.15.96:0/1656032782 conn(0x7f8fc4098a60 msgr2=0x7f8fc409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/1656032782 shutdown_connections 2024-09-13T17:42:48.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/1656032782 wait complete. 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 Processor -- start 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- start start 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc4132f50 con 0x7f8fc4134e70 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc9f3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc9f3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55726/0 (socket says 172.21.15.96:55726) 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc9f3e640 1 -- 172.21.15.96:0/3311057422 learned_addr learned my addr 172.21.15.96:0/3311057422 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc9f3e640 1 -- 172.21.15.96:0/3311057422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009420 con 0x7f8fc4134e70 2024-09-13T17:42:48.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc9f3e640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0019030 tx=0x7f8fc001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc00b7450 con 0x7f8fc4134e70 2024-09-13T17:42:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00b75b0 con 0x7f8fc4134e70 2024-09-13T17:42:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40a1ff0 con 0x7f8fc4134e70 2024-09-13T17:42:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8fc00b7890 con 0x7f8fc4134e70 2024-09-13T17:42:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc413ba70 con 0x7f8fc4134e70 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8fc00b79f0 con 0x7f8fc4134e70 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc41368b0 con 0x7f8fc4134e70 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 0x7f8fa8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f8fc00e1cb0 con 0x7f8fc4134e70 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc973d640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 0x7f8fa8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.139+0000 7f8fc973d640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 0x7f8fa8037910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f90 tx=0x7f8fbc0016a0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.143+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8fc00c74e0 con 0x7f8fc4134e70 2024-09-13T17:42:48.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc40a2740 con 0x7f8fc4134e70 2024-09-13T17:42:48.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fb67fc640 1 -- 172.21.15.96:0/3311057422 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+6723 (secure 0 0 0) 0x7f8fc00bb070 con 0x7f8fc4134e70 2024-09-13T17:42:48.298 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:48.299 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":21,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:45.492425+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:48.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 msgr2=0x7f8fa8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 0x7f8fa8037910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f90 tx=0x7f8fbc0016a0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 msgr2=0x7f8fc413aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0019030 tx=0x7f8fc001ac60 comp rx=0 tx=0).stop 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 shutdown_connections 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8fa8035450 0x7f8fa8037910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 --2- 172.21.15.96:0/3311057422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4134e70 0x7f8fc413aff0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 >> 172.21.15.96:0/3311057422 conn(0x7f8fc4098a60 msgr2=0x7f8fc4098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.295+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 shutdown_connections 2024-09-13T17:42:48.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.299+0000 7f8fcc1a2640 1 -- 172.21.15.96:0/3311057422 wait complete. 2024-09-13T17:42:48.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:42:48.316 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:42:48.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fff0a640 1 Processor -- start 2024-09-13T17:42:48.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fff0a640 1 -- start start 2024-09-13T17:42:48.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fff0a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f41f806f6d0 0x7f41f8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fff0a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f806e130 con 0x7f41f809dc20 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fff0a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f80a6750 con 0x7f41f806f6d0 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fdca6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f41f806f6d0 0x7f41f8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fd4a5640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f41f809dc20 0x7f41f8134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36090/0 (socket says 172.21.15.96:36090) 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fd4a5640 1 -- 172.21.15.96:0/1455603878 learned_addr learned my addr 172.21.15.96:0/1455603878 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fdca6640 1 -- 172.21.15.96:0/1455603878 >> v1:172.21.15.96:6789/0 conn(0x7f41f809dc20 legacy=0x7f41f8134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.423+0000 7f41fcca4640 1 -- 172.21.15.96:0/1455603878 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526819138 0 0) 0x7f41f806e130 con 0x7f41f809dc20 2024-09-13T17:42:48.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fdca6640 1 -- 172.21.15.96:0/1455603878 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f80712e0 con 0x7f41f806f6d0 2024-09-13T17:42:48.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fdca6640 1 --2- 172.21.15.96:0/1455603878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f8134ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009740 tx=0x7f41ec01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b0a424df10c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fcca4640 1 -- 172.21.15.96:0/1455603878 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41ec0ab400 con 0x7f41f806f6d0 2024-09-13T17:42:48.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fcca4640 1 -- 172.21.15.96:0/1455603878 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec0a7050 con 0x7f41f806f6d0 2024-09-13T17:42:48.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fcca4640 1 -- 172.21.15.96:0/1455603878 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41ec0ab6b0 con 0x7f41f806f6d0 2024-09-13T17:42:48.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 -- 172.21.15.96:0/1455603878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 msgr2=0x7f41f8134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1455603878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f8134ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009740 tx=0x7f41ec01c390 comp rx=0 tx=0).stop 2024-09-13T17:42:48.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 -- 172.21.15.96:0/1455603878 shutdown_connections 2024-09-13T17:42:48.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1455603878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f8134ab0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 -- 172.21.15.96:0/1455603878 >> 172.21.15.96:0/1455603878 conn(0x7f41f8098a60 msgr2=0x7f41f809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 -- 172.21.15.96:0/1455603878 shutdown_connections 2024-09-13T17:42:48.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.427+0000 7f41fff0a640 1 -- 172.21.15.96:0/1455603878 wait complete. 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 Processor -- start 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 -- start start 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f80a2090 con 0x7f41f806f6d0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fdca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fdca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55758/0 (socket says 172.21.15.96:55758) 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fdca6640 1 -- 172.21.15.96:0/1912226827 learned_addr learned my addr 172.21.15.96:0/1912226827 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fdca6640 1 -- 172.21.15.96:0/1912226827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec009420 con 0x7f41f806f6d0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fdca6640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f41ec019fc0 tx=0x7f41ec01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41ec0abea0 con 0x7f41f806f6d0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec004020 con 0x7f41f806f6d0 2024-09-13T17:42:48.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f80a38e0 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41ec0042c0 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f80a4da0 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f80a1d40 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f41ec0b2050 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 0x7f41dc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f41ec0e1690 con 0x7f41f806f6d0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fd4a5640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 0x7f41dc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.431+0000 7f41fd4a5640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 0x7f41dc037910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f41f4000f90 tx=0x7f41f4007ca0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.435+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41ec0b2300 con 0x7f41f806f6d0 2024-09-13T17:42:48.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41f8134ab0 con 0x7f41f806f6d0 2024-09-13T17:42:48.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41ea7fc640 1 -- 172.21.15.96:0/1912226827 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41ec0b0070 con 0x7f41f806f6d0 2024-09-13T17:42:48.586 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 msgr2=0x7f41dc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 0x7f41dc037910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f41f4000f90 tx=0x7f41f4007ca0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 msgr2=0x7f41f80a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f41ec019fc0 tx=0x7f41ec01abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 shutdown_connections 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f41dc035450 0x7f41dc037910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 --2- 172.21.15.96:0/1912226827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41f806f6d0 0x7f41f80a4630 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 >> 172.21.15.96:0/1912226827 conn(0x7f41f8098a60 msgr2=0x7f41f806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 shutdown_connections 2024-09-13T17:42:48.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.583+0000 7f41fff0a640 1 -- 172.21.15.96:0/1912226827 wait complete. 2024-09-13T17:42:48.604 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:42:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.711+0000 7ff0e8043640 1 Processor -- start 2024-09-13T17:42:48.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.711+0000 7ff0e8043640 1 -- start start 2024-09-13T17:42:48.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0e009d690 0x7ff0e0138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e0138050 con 0x7ff0e01387d0 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e009c410 con 0x7ff0e009d690 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e5ddf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0e009d690 0x7ff0e0138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e55de640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff0e01387d0 0x7ff0e0134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36102/0 (socket says 172.21.15.96:36102) 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e5ddf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff0e009d690 0x7ff0e0138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55764/0 (socket says 172.21.15.96:55764) 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e55de640 1 -- 172.21.15.96:0/688283764 learned_addr learned my addr 172.21.15.96:0/688283764 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e5ddf640 1 -- 172.21.15.96:0/688283764 >> v1:172.21.15.96:6789/0 conn(0x7ff0e01387d0 legacy=0x7ff0e0134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e5ddf640 1 -- 172.21.15.96:0/688283764 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e009cba0 con 0x7ff0e009d690 2024-09-13T17:42:48.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e5ddf640 1 --2- 172.21.15.96:0/688283764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e0138290 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009740 tx=0x7ff0dc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54db07d49c54239 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e4ddd640 1 -- 172.21.15.96:0/688283764 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc0a8840 con 0x7ff0e009d690 2024-09-13T17:42:48.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e4ddd640 1 -- 172.21.15.96:0/688283764 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0a89a0 con 0x7ff0e009d690 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e4ddd640 1 -- 172.21.15.96:0/688283764 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc0a8c60 con 0x7ff0e009d690 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- 172.21.15.96:0/688283764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 msgr2=0x7ff0e0138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 --2- 172.21.15.96:0/688283764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e0138290 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc009740 tx=0x7ff0dc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- 172.21.15.96:0/688283764 shutdown_connections 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 --2- 172.21.15.96:0/688283764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e0138290 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- 172.21.15.96:0/688283764 >> 172.21.15.96:0/688283764 conn(0x7ff0e006bc60 msgr2=0x7ff0e006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- 172.21.15.96:0/688283764 shutdown_connections 2024-09-13T17:42:48.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.715+0000 7ff0e8043640 1 -- 172.21.15.96:0/688283764 wait complete. 2024-09-13T17:42:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 Processor -- start 2024-09-13T17:42:48.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 -- start start 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e00a5d10 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e5ddf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e5ddf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55768/0 (socket says 172.21.15.96:55768) 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e5ddf640 1 -- 172.21.15.96:0/3083702825 learned_addr learned my addr 172.21.15.96:0/3083702825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e5ddf640 1 -- 172.21.15.96:0/3083702825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0dc009420 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e5ddf640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc01c840 tx=0x7ff0dc01b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc0b0840 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0dc0b09a0 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e00a4470 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff0dc0b0c60 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e009f490 con 0x7ff0e009d690 2024-09-13T17:42:48.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7ff0dc0b7040 con 0x7ff0e009d690 2024-09-13T17:42:48.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e009bc00 con 0x7ff0e009d690 2024-09-13T17:42:48.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 0x7ff0c4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:48.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7ff0dc0e5000 con 0x7ff0e009d690 2024-09-13T17:42:48.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e55de640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 0x7ff0c4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:48.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.719+0000 7ff0e55de640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 0x7ff0c4037910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005890 tx=0x7ff0d0007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:48.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.723+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff0e009bc00 con 0x7ff0e009d690 2024-09-13T17:42:48.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.883+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff0e009bc00 con 0x7ff0e009d690 2024-09-13T17:42:48.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.883+0000 7ff0d67fc640 1 -- 172.21.15.96:0/3083702825 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v21) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff0e009bc00 con 0x7ff0e009d690 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:42:48.889 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 msgr2=0x7ff0c4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 0x7ff0c4037910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0005890 tx=0x7ff0d0007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 msgr2=0x7ff0e00a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc01c840 tx=0x7ff0dc01b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 shutdown_connections 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7ff0c4035450 0x7ff0c4037910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 --2- 172.21.15.96:0/3083702825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0e009d690 0x7ff0e00a6a60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 >> 172.21.15.96:0/3083702825 conn(0x7ff0e006bc60 msgr2=0x7ff0e0099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 shutdown_connections 2024-09-13T17:42:48.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:48.887+0000 7ff0e8043640 1 -- 172.21.15.96:0/3083702825 wait complete. 2024-09-13T17:42:48.907 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:42:48.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:42:49.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4098b71640 1 Processor -- start 2024-09-13T17:42:49.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4098b71640 1 -- start start 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4098b71640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4094133050 0x7f40941323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4098b71640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4094132a70 con 0x7f4094132c70 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4098b71640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409409bb70 con 0x7f4094133050 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4091d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4094133050 0x7f40941323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4091d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4094133050 0x7f40941323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55778/0 (socket says 172.21.15.96:55778) 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f409259c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4094132c70 0x7f40941359d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36112/0 (socket says 172.21.15.96:36112) 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4091d9b640 1 -- 172.21.15.96:0/2580370859 learned_addr learned my addr 172.21.15.96:0/2580370859 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4091d9b640 1 -- 172.21.15.96:0/2580370859 >> v1:172.21.15.96:6789/0 conn(0x7f4094132c70 legacy=0x7f40941359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.027+0000 7f4091d9b640 1 -- 172.21.15.96:0/2580370859 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409409cbf0 con 0x7f4094133050 2024-09-13T17:42:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4091d9b640 1 --2- 172.21.15.96:0/2580370859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094133050 0x7f40941323e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f407c0097f0 tx=0x7f407c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db8bf4f394d11c39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409159a640 1 -- 172.21.15.96:0/2580370859 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f407c0ab530 con 0x7f4094133050 2024-09-13T17:42:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409159a640 1 -- 172.21.15.96:0/2580370859 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c0a7050 con 0x7f4094133050 2024-09-13T17:42:49.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409159a640 1 -- 172.21.15.96:0/2580370859 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f407c0ab7e0 con 0x7f4094133050 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2580370859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094133050 msgr2=0x7f40941323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 --2- 172.21.15.96:0/2580370859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094133050 0x7f40941323e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f407c0097f0 tx=0x7f407c01c450 comp rx=0 tx=0).stop 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2580370859 shutdown_connections 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 --2- 172.21.15.96:0/2580370859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094133050 0x7f40941323e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2580370859 >> 172.21.15.96:0/2580370859 conn(0x7f409406f0a0 msgr2=0x7f40940714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2580370859 shutdown_connections 2024-09-13T17:42:49.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2580370859 wait complete. 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 Processor -- start 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- start start 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409406c5f0 con 0x7f4094132c70 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55784/0 (socket says 172.21.15.96:55784) 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409259c640 1 -- 172.21.15.96:0/2213097201 learned_addr learned my addr 172.21.15.96:0/2213097201 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409259c640 1 -- 172.21.15.96:0/2213097201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c0094d0 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f409259c640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c7f0 tx=0x7f408c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f408c00a610 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408c00a770 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409406ce60 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f408c00aa10 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409406cae0 con 0x7f4094132c70 2024-09-13T17:42:49.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.031+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409406cc70 con 0x7f4094132c70 2024-09-13T17:42:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.035+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f408c00ab70 con 0x7f4094132c70 2024-09-13T17:42:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.035+0000 7f408affd640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 0x7f4074037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.035+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f408c047060 con 0x7f4094132c70 2024-09-13T17:42:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.035+0000 7f4091d9b640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 0x7f4074037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.035+0000 7f4091d9b640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 0x7f4074037910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f407c0ad000 tx=0x7f407c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.039+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f408c007070 con 0x7f4094132c70 2024-09-13T17:42:49.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.183+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f40941323e0 con 0x7f4094132c70 2024-09-13T17:42:49.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.663+0000 7f408affd640 1 -- 172.21.15.96:0/2213097201 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v22) v1 ==== 138+0+0 (secure 0 0 0) 0x7f408c017030 con 0x7f4094132c70 2024-09-13T17:42:49.669 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 msgr2=0x7f4074037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 0x7f4074037910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f407c0ad000 tx=0x7f407c0a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 msgr2=0x7f409406bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f408c00c7f0 tx=0x7f408c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 shutdown_connections 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4074035450 0x7f4074037910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:49.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 --2- 172.21.15.96:0/2213097201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4094132c70 0x7f409406bd90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:49.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 >> 172.21.15.96:0/2213097201 conn(0x7f409406f0a0 msgr2=0x7f409412f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:49.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 shutdown_connections 2024-09-13T17:42:49.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.671+0000 7f4098b71640 1 -- 172.21.15.96:0/2213097201 wait complete. 2024-09-13T17:42:49.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:42:49.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc56962640 1 Processor -- start 2024-09-13T17:42:49.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc56962640 1 -- start start 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc56962640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbc5009d690 0x7fbc50138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc56962640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc50138050 con 0x7fbc501387d0 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc56962640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5009c410 con 0x7fbc5009d690 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbc5009d690 0x7fbc50138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbc501387d0 0x7fbc50134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36124/0 (socket says 172.21.15.96:36124) 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbc5009d690 0x7fbc50138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55790/0 (socket says 172.21.15.96:55790) 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4f7fe640 1 -- 172.21.15.96:0/59253442 learned_addr learned my addr 172.21.15.96:0/59253442 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4ffff640 1 -- 172.21.15.96:0/59253442 >> v1:172.21.15.96:6789/0 conn(0x7fbc501387d0 legacy=0x7fbc50134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.819+0000 7fbc4ffff640 1 -- 172.21.15.96:0/59253442 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5009cba0 con 0x7fbc5009d690 2024-09-13T17:42:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 --2- 172.21.15.96:0/59253442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc50138290 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009740 tx=0x7fbc4001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf4e30d38fd6109 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4effd640 1 -- 172.21.15.96:0/59253442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc400a8840 con 0x7fbc5009d690 2024-09-13T17:42:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4effd640 1 -- 172.21.15.96:0/59253442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc400a89a0 con 0x7fbc5009d690 2024-09-13T17:42:49.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4effd640 1 -- 172.21.15.96:0/59253442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc400a8c60 con 0x7fbc5009d690 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/59253442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 msgr2=0x7fbc50138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 --2- 172.21.15.96:0/59253442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc50138290 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009740 tx=0x7fbc4001ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/59253442 shutdown_connections 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 --2- 172.21.15.96:0/59253442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc50138290 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/59253442 >> 172.21.15.96:0/59253442 conn(0x7fbc5006bc60 msgr2=0x7fbc5006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/59253442 shutdown_connections 2024-09-13T17:42:49.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/59253442 wait complete. 2024-09-13T17:42:49.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 Processor -- start 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- start start 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc500a5e10 con 0x7fbc5009d690 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55804/0 (socket says 172.21.15.96:55804) 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 -- 172.21.15.96:0/3484000316 learned_addr learned my addr 172.21.15.96:0/3484000316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 -- 172.21.15.96:0/3484000316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc40009420 con 0x7fbc5009d690 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4ffff640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001c840 tx=0x7fbc4001b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc400b0840 con 0x7fbc5009d690 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc400b09a0 con 0x7fbc5009d690 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc500a70a0 con 0x7fbc5009d690 2024-09-13T17:42:49.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc400b0c60 con 0x7fbc5009d690 2024-09-13T17:42:49.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc500a4300 con 0x7fbc5009d690 2024-09-13T17:42:49.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5009bc00 con 0x7fbc5009d690 2024-09-13T17:42:49.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.823+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fbc400b7040 con 0x7fbc5009d690 2024-09-13T17:42:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.827+0000 7fbc4cff9640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 0x7fbc3c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.827+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 2957+0+0 (secure 0 0 0) 0x7fbc400e5170 con 0x7fbc5009d690 2024-09-13T17:42:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.827+0000 7fbc4f7fe640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 0x7fbc3c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:49.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.827+0000 7fbc4f7fe640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 0x7fbc3c037910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005890 tx=0x7fbc44007a90 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:49.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.831+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbc5009bc00 con 0x7fbc5009d690 2024-09-13T17:42:49.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:49.987+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbc5009bc00 con 0x7fbc5009d690 2024-09-13T17:42:50.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.679+0000 7fbc4cff9640 1 -- 172.21.15.96:0/3484000316 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v23) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbc5009bc00 con 0x7fbc5009d690 2024-09-13T17:42:50.682 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-13T17:42:50.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.687+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 msgr2=0x7fbc3c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:50.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.687+0000 7fbc56962640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 0x7fbc3c037910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbc44005890 tx=0x7fbc44007a90 comp rx=0 tx=0).stop 2024-09-13T17:42:50.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.687+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 msgr2=0x7fbc500a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:50.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.687+0000 7fbc56962640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbc4001c840 tx=0x7fbc4001b9b0 comp rx=0 tx=0).stop 2024-09-13T17:42:50.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 shutdown_connections 2024-09-13T17:42:50.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fbc3c035450 0x7fbc3c037910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:50.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 --2- 172.21.15.96:0/3484000316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc5009d690 0x7fbc500a6b60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:50.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 >> 172.21.15.96:0/3484000316 conn(0x7fbc5006bc60 msgr2=0x7fbc501bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:50.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 shutdown_connections 2024-09-13T17:42:50.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.691+0000 7fbc56962640 1 -- 172.21.15.96:0/3484000316 wait complete. 2024-09-13T17:42:50.727 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:42:50.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.843+0000 7f610f8bb640 1 Processor -- start 2024-09-13T17:42:50.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.843+0000 7f610f8bb640 1 -- start start 2024-09-13T17:42:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f610813af50 0x7f610813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6108138050 con 0x7f610809d690 2024-09-13T17:42:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610809c410 con 0x7f610813af50 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610ce56640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f610813af50 0x7f610813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610d657640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f610809d690 0x7f6108138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36126/0 (socket says 172.21.15.96:36126) 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610ce56640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f610813af50 0x7f610813d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55818/0 (socket says 172.21.15.96:55818) 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610d657640 1 -- 172.21.15.96:0/2291996539 learned_addr learned my addr 172.21.15.96:0/2291996539 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610ce56640 1 -- 172.21.15.96:0/2291996539 >> v1:172.21.15.96:6789/0 conn(0x7f610809d690 legacy=0x7f6108138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610ce56640 1 -- 172.21.15.96:0/2291996539 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610809cba0 con 0x7f610813af50 2024-09-13T17:42:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f60fffff640 1 -- 172.21.15.96:0/2291996539 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739023583 0 0) 0x7f61040014a0 con 0x7f610809d690 2024-09-13T17:42:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610ce56640 1 --2- 172.21.15.96:0/2291996539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610813af50 0x7f610813d340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009740 tx=0x7f60f801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c7d9a306240b8e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f60fffff640 1 -- 172.21.15.96:0/2291996539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60f80a8840 con 0x7f610813af50 2024-09-13T17:42:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f60fffff640 1 -- 172.21.15.96:0/2291996539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f80a89a0 con 0x7f610813af50 2024-09-13T17:42:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f60fffff640 1 -- 172.21.15.96:0/2291996539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60f80a8c60 con 0x7f610813af50 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- 172.21.15.96:0/2291996539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610813af50 msgr2=0x7f610813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 --2- 172.21.15.96:0/2291996539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610813af50 0x7f610813d340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f60f8009740 tx=0x7f60f801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- 172.21.15.96:0/2291996539 shutdown_connections 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 --2- 172.21.15.96:0/2291996539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610813af50 0x7f610813d340 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- 172.21.15.96:0/2291996539 >> 172.21.15.96:0/2291996539 conn(0x7f610806bc60 msgr2=0x7f610806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- 172.21.15.96:0/2291996539 shutdown_connections 2024-09-13T17:42:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.847+0000 7f610f8bb640 1 -- 172.21.15.96:0/2291996539 wait complete. 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 Processor -- start 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 -- start start 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61080a4c60 con 0x7f610809d690 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610d657640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610d657640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55832/0 (socket says 172.21.15.96:55832) 2024-09-13T17:42:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610d657640 1 -- 172.21.15.96:0/1961964259 learned_addr learned my addr 172.21.15.96:0/1961964259 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610d657640 1 -- 172.21.15.96:0/1961964259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8009420 con 0x7f610809d690 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610d657640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f610400c700 tx=0x7f610400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6104004d80 con 0x7f610809d690 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610400a660 con 0x7f610809d690 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61080a5ee0 con 0x7f610809d690 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f610400a940 con 0x7f610809d690 2024-09-13T17:42:50.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61081372d0 con 0x7f610809d690 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610809bc00 con 0x7f610809d690 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f610400aaa0 con 0x7f610809d690 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 0x7f60ec037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3339+0+0 (secure 0 0 0) 0x7f6104046d40 con 0x7f610809d690 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610ce56640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 0x7f60ec037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.851+0000 7f610ce56640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 0x7f60ec037910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60f80b2000 tx=0x7f60f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:50.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:50.859+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f610809bc00 con 0x7f610809d690 2024-09-13T17:42:51.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.051+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6108138050 con 0x7f610809d690 2024-09-13T17:42:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.087+0000 7f60fdffb640 1 -- 172.21.15.96:0/1961964259 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6108138050 con 0x7f610809d690 2024-09-13T17:42:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 4 and data pool 5 2024-09-13T17:42:51.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 msgr2=0x7f60ec037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 0x7f60ec037910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f60f80b2000 tx=0x7f60f8008040 comp rx=0 tx=0).stop 2024-09-13T17:42:51.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 msgr2=0x7f61080a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f610400c700 tx=0x7f610400cbc0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 shutdown_connections 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f60ec035450 0x7f60ec037910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 --2- 172.21.15.96:0/1961964259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f610809d690 0x7f61080a6900 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 >> 172.21.15.96:0/1961964259 conn(0x7f610806bc60 msgr2=0x7f6108139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 shutdown_connections 2024-09-13T17:42:51.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.091+0000 7f610f8bb640 1 -- 172.21.15.96:0/1961964259 wait complete. 2024-09-13T17:42:51.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f24231e7640 1 Processor -- start 2024-09-13T17:42:51.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f24231e7640 1 -- start start 2024-09-13T17:42:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f24231e7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f241c134e70 0x7f241c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f24231e7640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c09ed60 con 0x7f241c135250 2024-09-13T17:42:51.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f24231e7640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c09ddc0 con 0x7f241c134e70 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f241bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f241c135250 0x7f241c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36132/0 (socket says 172.21.15.96:36132) 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f241bfff640 1 -- 172.21.15.96:0/1013768965 learned_addr learned my addr 172.21.15.96:0/1013768965 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f2420f83640 1 --2- 172.21.15.96:0/1013768965 >> v2:172.21.15.96:3300/0 conn(0x7f241c134e70 0x7f241c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136322831 0 0) 0x7f240c001120 con 0x7f241c135250 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2404003530 con 0x7f241c135250 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f2420f83640 1 -- 172.21.15.96:0/1013768965 >> v1:172.21.15.96:6789/0 conn(0x7f241c135250 legacy=0x7f241c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065331505 0 0) 0x7f2404003530 con 0x7f241c135250 2024-09-13T17:42:51.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.231+0000 7f2420f83640 1 -- 172.21.15.96:0/1013768965 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0a68b0 con 0x7f241c134e70 2024-09-13T17:42:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f2420f83640 1 --2- 172.21.15.96:0/1013768965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c09e820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2410009740 tx=0x7f241001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850e1dc42f523aae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f240c001740 con 0x7f241c134e70 2024-09-13T17:42:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24100a7050 con 0x7f241c134e70 2024-09-13T17:42:51.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f241b7fe640 1 -- 172.21.15.96:0/1013768965 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24100ab530 con 0x7f241c134e70 2024-09-13T17:42:51.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f24231e7640 1 -- 172.21.15.96:0/1013768965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 msgr2=0x7f241c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.235+0000 7f24231e7640 1 --2- 172.21.15.96:0/1013768965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c09e820 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2410009740 tx=0x7f241001c340 comp rx=0 tx=0).stop 2024-09-13T17:42:51.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1013768965 shutdown_connections 2024-09-13T17:42:51.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 --2- 172.21.15.96:0/1013768965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c09e820 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1013768965 >> 172.21.15.96:0/1013768965 conn(0x7f241c098a60 msgr2=0x7f241c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1013768965 shutdown_connections 2024-09-13T17:42:51.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1013768965 wait complete. 2024-09-13T17:42:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 Processor -- start 2024-09-13T17:42:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- start start 2024-09-13T17:42:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c132f50 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f2420f83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f2420f83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55858/0 (socket says 172.21.15.96:55858) 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f2420f83640 1 -- 172.21.15.96:0/1828959976 learned_addr learned my addr 172.21.15.96:0/1828959976 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f2420f83640 1 -- 172.21.15.96:0/1828959976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410009420 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f2420f83640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f24100ae040 tx=0x7f241001ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24100abd70 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24100b7450 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0a0a40 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24100b75c0 con 0x7f241c134e70 2024-09-13T17:42:51.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c0a0570 con 0x7f241c134e70 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f24100b7720 con 0x7f241c134e70 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c09e820 con 0x7f241c134e70 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 0x7f2408037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f24100e1ee0 con 0x7f241c134e70 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.239+0000 7f241bfff640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 0x7f2408037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.243+0000 7f241bfff640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 0x7f2408037910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f240c000c10 tx=0x7f240c001db0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.247+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f24100c76c0 con 0x7f241c134e70 2024-09-13T17:42:51.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.403+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f241c0a2820 con 0x7f241c134e70 2024-09-13T17:42:51.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.407+0000 7f24197fa640 1 -- 172.21.15.96:0/1828959976 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10504 (secure 0 0 0) 0x7f24100bb070 con 0x7f241c134e70 2024-09-13T17:42:51.411 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:51.411 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:51.057554+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:51.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 msgr2=0x7f2408037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 0x7f2408037910 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f240c000c10 tx=0x7f240c001db0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 msgr2=0x7f241c0a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f24100ae040 tx=0x7f241001ab80 comp rx=0 tx=0).stop 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 shutdown_connections 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2408035450 0x7f2408037910 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 --2- 172.21.15.96:0/1828959976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c134e70 0x7f241c0a1070 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 >> 172.21.15.96:0/1828959976 conn(0x7f241c098a60 msgr2=0x7f241c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 shutdown_connections 2024-09-13T17:42:51.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.411+0000 7f24231e7640 1 -- 172.21.15.96:0/1828959976 wait complete. 2024-09-13T17:42:51.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:51.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.547+0000 7f8e0bf22640 1 Processor -- start 2024-09-13T17:42:51.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.547+0000 7f8e0bf22640 1 -- start start 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e0bf22640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e041329b0 0x7f8e04132d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e0bf22640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e041327b0 con 0x7f8e0409dc30 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e0bf22640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0409bd90 con 0x7f8e041329b0 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e041329b0 0x7f8e04132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8e041329b0 0x7f8e04132d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55866/0 (socket says 172.21.15.96:55866) 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 -- 172.21.15.96:0/1888991404 learned_addr learned my addr 172.21.15.96:0/1888991404 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e094bd640 1 --1- 172.21.15.96:0/1888991404 >> v1:172.21.15.96:6789/0 conn(0x7f8e0409dc30 0x7f8e04135980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36148/0 (socket says 172.21.15.96:36148) 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 -- 172.21.15.96:0/1888991404 >> v1:172.21.15.96:6789/0 conn(0x7f8e0409dc30 legacy=0x7f8e04135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 -- 172.21.15.96:0/1888991404 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0409b2a0 con 0x7f8e041329b0 2024-09-13T17:42:51.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e09cbe640 1 --2- 172.21.15.96:0/1888991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e041329b0 0x7f8e04132d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e000097f0 tx=0x7f8e0001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef55c13c14386 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e08cbc640 1 -- 172.21.15.96:0/1888991404 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e000ab530 con 0x7f8e041329b0 2024-09-13T17:42:51.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e08cbc640 1 -- 172.21.15.96:0/1888991404 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e000a7050 con 0x7f8e041329b0 2024-09-13T17:42:51.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.551+0000 7f8e08cbc640 1 -- 172.21.15.96:0/1888991404 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e000ab7e0 con 0x7f8e041329b0 2024-09-13T17:42:51.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1888991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e041329b0 msgr2=0x7f8e04132d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1888991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e041329b0 0x7f8e04132d90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f8e000097f0 tx=0x7f8e0001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1888991404 shutdown_connections 2024-09-13T17:42:51.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1888991404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e041329b0 0x7f8e04132d90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1888991404 >> 172.21.15.96:0/1888991404 conn(0x7f8e0406ef80 msgr2=0x7f8e040713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1888991404 shutdown_connections 2024-09-13T17:42:51.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1888991404 wait complete. 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 Processor -- start 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- start start 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e040a0030 con 0x7f8e0409dc30 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e09cbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e09cbe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55876/0 (socket says 172.21.15.96:55876) 2024-09-13T17:42:51.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e09cbe640 1 -- 172.21.15.96:0/1904850115 learned_addr learned my addr 172.21.15.96:0/1904850115 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e09cbe640 1 -- 172.21.15.96:0/1904850115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000094d0 con 0x7f8e0409dc30 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e09cbe640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8e000190f0 tx=0x7f8e0001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e000b6420 con 0x7f8e0409dc30 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e000b65a0 con 0x7f8e0409dc30 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e04139fe0 con 0x7f8e0409dc30 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8e000b6880 con 0x7f8e0409dc30 2024-09-13T17:42:51.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0413b830 con 0x7f8e0409dc30 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f8e000b2040 con 0x7f8e0409dc30 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e040008e0 con 0x7f8e0409dc30 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 0x7f8de8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8e000e1b50 con 0x7f8e0409dc30 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.555+0000 7f8e094bd640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 0x7f8de8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.559+0000 7f8e094bd640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 0x7f8de8037910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e0409e040 tx=0x7f8df4007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.563+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8e000b69e0 con 0x7f8e0409dc30 2024-09-13T17:42:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.711+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e0409d340 con 0x7f8e0409dc30 2024-09-13T17:42:51.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.711+0000 7f8dfa7fc640 1 -- 172.21.15.96:0/1904850115 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+10504 (secure 0 0 0) 0x7f8e000b0030 con 0x7f8e0409dc30 2024-09-13T17:42:51.716 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:51.716 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":24,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:51.057554+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 msgr2=0x7f8de8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 0x7f8de8037910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e0409e040 tx=0x7f8df4007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 msgr2=0x7f8e041380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f8e000190f0 tx=0x7f8e0001ad20 comp rx=0 tx=0).stop 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 shutdown_connections 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f8de8035450 0x7f8de8037910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 --2- 172.21.15.96:0/1904850115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e0409dc30 0x7f8e041380c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 >> 172.21.15.96:0/1904850115 conn(0x7f8e0406ef80 msgr2=0x7f8e0412f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 shutdown_connections 2024-09-13T17:42:51.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.715+0000 7f8e0bf22640 1 -- 172.21.15.96:0/1904850115 wait complete. 2024-09-13T17:42:51.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:42:51.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.851+0000 7fd2ff48b640 1 Processor -- start 2024-09-13T17:42:51.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2ff48b640 1 -- start start 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2ff48b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2f8065c80 0x7fd2f81318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2ff48b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f806f500 con 0x7fd2f8070400 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2ff48b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f806f810 con 0x7fd2f8065c80 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2f8065c80 0x7fd2f81318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2f8065c80 0x7fd2f81318a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55882/0 (socket says 172.21.15.96:55882) 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fca26640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd2f8070400 0x7fd2f8131de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36150/0 (socket says 172.21.15.96:36150) 2024-09-13T17:42:51.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 -- 172.21.15.96:0/2913833617 learned_addr learned my addr 172.21.15.96:0/2913833617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:51.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 -- 172.21.15.96:0/2913833617 >> v1:172.21.15.96:6789/0 conn(0x7fd2f8070400 legacy=0x7fd2f8131de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2effff640 1 -- 172.21.15.96:0/2913833617 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 752337079 0 0) 0x7fd2f806f500 con 0x7fd2f8070400 2024-09-13T17:42:51.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 -- 172.21.15.96:0/2913833617 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f80a28b0 con 0x7fd2f8065c80 2024-09-13T17:42:51.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2fd227640 1 --2- 172.21.15.96:0/2913833617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f81318a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ba90 tx=0x7fd2f401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656739a6a2588de3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2effff640 1 -- 172.21.15.96:0/2913833617 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2f40a7800 con 0x7fd2f8065c80 2024-09-13T17:42:51.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2effff640 1 -- 172.21.15.96:0/2913833617 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f40a7960 con 0x7fd2f8065c80 2024-09-13T17:42:51.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.855+0000 7fd2effff640 1 -- 172.21.15.96:0/2913833617 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2f40a7c20 con 0x7fd2f8065c80 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/2913833617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 msgr2=0x7fd2f81318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/2913833617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f81318a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400ba90 tx=0x7fd2f401eb00 comp rx=0 tx=0).stop 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/2913833617 shutdown_connections 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/2913833617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f81318a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/2913833617 >> 172.21.15.96:0/2913833617 conn(0x7fd2f806a150 msgr2=0x7fd2f806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:51.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/2913833617 shutdown_connections 2024-09-13T17:42:51.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/2913833617 wait complete. 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 Processor -- start 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- start start 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f809dc60 con 0x7fd2f8065c80 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fd227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fd227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55892/0 (socket says 172.21.15.96:55892) 2024-09-13T17:42:51.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fd227640 1 -- 172.21.15.96:0/992936569 learned_addr learned my addr 172.21.15.96:0/992936569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fd227640 1 -- 172.21.15.96:0/992936569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f400b770 con 0x7fd2f8065c80 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fd227640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b2040 tx=0x7fd2f401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2f4004150 con 0x7fd2f8065c80 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f40042b0 con 0x7fd2f8065c80 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f8137870 con 0x7fd2f8065c80 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2f40ba660 con 0x7fd2f8065c80 2024-09-13T17:42:51.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f809d500 con 0x7fd2f8065c80 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd2f40b6070 con 0x7fd2f8065c80 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f8136e40 con 0x7fd2f8065c80 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 0x7fd2dc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fd2f40e2550 con 0x7fd2f8065c80 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fca26640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 0x7fd2dc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:51.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.859+0000 7fd2fca26640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 0x7fd2dc037910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007fd0 tx=0x7fd2e8007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:51.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:51.867+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2f40c2970 con 0x7fd2f8065c80 2024-09-13T17:42:52.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:52.055+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd2f80a5150 con 0x7fd2f8065c80 2024-09-13T17:42:53.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.063+0000 7fd2edffb640 1 -- 172.21.15.96:0/992936569 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd2f40be070 con 0x7fd2f8065c80 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 msgr2=0x7fd2dc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 0x7fd2dc037910 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8007fd0 tx=0x7fd2e8007920 comp rx=0 tx=0).stop 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 msgr2=0x7fd2f809df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40b2040 tx=0x7fd2f401d340 comp rx=0 tx=0).stop 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 shutdown_connections 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd2dc035450 0x7fd2dc037910 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 --2- 172.21.15.96:0/992936569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2f8065c80 0x7fd2f809df60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 >> 172.21.15.96:0/992936569 conn(0x7fd2f806a150 msgr2=0x7fd2f812e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:53.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 shutdown_connections 2024-09-13T17:42:53.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.067+0000 7fd2ff48b640 1 -- 172.21.15.96:0/992936569 wait complete. 2024-09-13T17:42:53.093 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:42:53.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:42:53.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.215+0000 7f81d0efb640 1 Processor -- start 2024-09-13T17:42:53.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.215+0000 7f81d0efb640 1 -- start start 2024-09-13T17:42:53.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f81cc135270 0x7f81cc13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc070de0 con 0x7f81cc134e90 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc071d60 con 0x7f81cc135270 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c9d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f81cc135270 0x7f81cc13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81ca59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f81cc134e90 0x7f81cc071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44308/0 (socket says 172.21.15.96:44308) 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81ca59c640 1 -- 172.21.15.96:0/4254139695 learned_addr learned my addr 172.21.15.96:0/4254139695 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c9d9b640 1 -- 172.21.15.96:0/4254139695 >> v1:172.21.15.96:6789/0 conn(0x7f81cc134e90 legacy=0x7f81cc071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c9d9b640 1 -- 172.21.15.96:0/4254139695 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc06f110 con 0x7f81cc135270 2024-09-13T17:42:53.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c959a640 1 -- 172.21.15.96:0/4254139695 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741106457 0 0) 0x7f81cc070de0 con 0x7f81cc134e90 2024-09-13T17:42:53.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c9d9b640 1 --2- 172.21.15.96:0/4254139695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc135270 0x7f81cc13cfe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009740 tx=0x7f81b401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f99e9a82545215 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:53.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c959a640 1 -- 172.21.15.96:0/4254139695 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81b40ab530 con 0x7f81cc135270 2024-09-13T17:42:53.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c959a640 1 -- 172.21.15.96:0/4254139695 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b40a7050 con 0x7f81cc135270 2024-09-13T17:42:53.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81c959a640 1 -- 172.21.15.96:0/4254139695 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81b40ab7e0 con 0x7f81cc135270 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- 172.21.15.96:0/4254139695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc135270 msgr2=0x7f81cc13cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 --2- 172.21.15.96:0/4254139695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc135270 0x7f81cc13cfe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009740 tx=0x7f81b401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- 172.21.15.96:0/4254139695 shutdown_connections 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 --2- 172.21.15.96:0/4254139695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc135270 0x7f81cc13cfe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- 172.21.15.96:0/4254139695 >> 172.21.15.96:0/4254139695 conn(0x7f81cc098a60 msgr2=0x7f81cc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- 172.21.15.96:0/4254139695 shutdown_connections 2024-09-13T17:42:53.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.219+0000 7f81d0efb640 1 -- 172.21.15.96:0/4254139695 wait complete. 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 Processor -- start 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 -- start start 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc09a9c0 con 0x7f81cc134e90 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81ca59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38806/0 (socket says 172.21.15.96:38806) 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81ca59c640 1 -- 172.21.15.96:0/2732235817 learned_addr learned my addr 172.21.15.96:0/2732235817 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81ca59c640 1 -- 172.21.15.96:0/2732235817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4009420 con 0x7f81cc134e90 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81ca59c640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c840 tx=0x7f81c400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:53.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81c400a660 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c400a7e0 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc11e370 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81c400aac0 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc0a54a0 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f81c400ac20 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc11d0e0 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 0x7f81ac037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f81c4046fe0 con 0x7f81cc134e90 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c9d9b640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 0x7f81ac037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:53.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.223+0000 7f81c9d9b640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 0x7f81ac037910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f81b40ad000 tx=0x7f81b40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:53.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.227+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81c4078db0 con 0x7f81cc134e90 2024-09-13T17:42:53.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:53.407+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f81cc071530 con 0x7f81cc134e90 2024-09-13T17:42:54.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.067+0000 7f81c2ffd640 1 -- 172.21.15.96:0/2732235817 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81c401a6b0 con 0x7f81cc134e90 2024-09-13T17:42:54.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 msgr2=0x7f81ac037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 0x7f81ac037910 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f81b40ad000 tx=0x7f81b40a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:54.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 msgr2=0x7f81cc11de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f81c400c840 tx=0x7f81c400cd00 comp rx=0 tx=0).stop 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 shutdown_connections 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f81ac035450 0x7f81ac037910 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 --2- 172.21.15.96:0/2732235817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81cc134e90 0x7f81cc11de30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 >> 172.21.15.96:0/2732235817 conn(0x7f81cc098a60 msgr2=0x7f81cc136f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 shutdown_connections 2024-09-13T17:42:54.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.071+0000 7f81d0efb640 1 -- 172.21.15.96:0/2732235817 wait complete. 2024-09-13T17:42:54.095 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:42:54.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:42:54.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.215+0000 7f7d394ca640 1 Processor -- start 2024-09-13T17:42:54.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.215+0000 7f7d394ca640 1 -- start start 2024-09-13T17:42:54.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7d34134e70 0x7f7d3409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3409ed60 con 0x7f7d34135250 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3409ddc0 con 0x7f7d34134e70 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7d34134e70 0x7f7d3409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7d34134e70 0x7f7d3409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38822/0 (socket says 172.21.15.96:38822) 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 -- 172.21.15.96:0/706078677 learned_addr learned my addr 172.21.15.96:0/706078677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 -- 172.21.15.96:0/706078677 >> v1:172.21.15.96:6789/0 conn(0x7f7d34135250 legacy=0x7f7d34135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d31ffb640 1 -- 172.21.15.96:0/706078677 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929324559 0 0) 0x7f7d28001120 con 0x7f7d34135250 2024-09-13T17:42:54.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 -- 172.21.15.96:0/706078677 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340a68b0 con 0x7f7d34134e70 2024-09-13T17:42:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d32ffd640 1 --2- 172.21.15.96:0/706078677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d3409e820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097f0 tx=0x7f7d2401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60edbe94ad40c018 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d31ffb640 1 -- 172.21.15.96:0/706078677 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d240ac540 con 0x7f7d34134e70 2024-09-13T17:42:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d31ffb640 1 -- 172.21.15.96:0/706078677 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240ac6a0 con 0x7f7d34134e70 2024-09-13T17:42:54.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d31ffb640 1 -- 172.21.15.96:0/706078677 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d240ac960 con 0x7f7d34134e70 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- 172.21.15.96:0/706078677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 msgr2=0x7f7d3409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 --2- 172.21.15.96:0/706078677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d3409e820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7d240097f0 tx=0x7f7d2401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- 172.21.15.96:0/706078677 shutdown_connections 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 --2- 172.21.15.96:0/706078677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d3409e820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- 172.21.15.96:0/706078677 >> 172.21.15.96:0/706078677 conn(0x7f7d34098a60 msgr2=0x7f7d3409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- 172.21.15.96:0/706078677 shutdown_connections 2024-09-13T17:42:54.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.219+0000 7f7d394ca640 1 -- 172.21.15.96:0/706078677 wait complete. 2024-09-13T17:42:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 Processor -- start 2024-09-13T17:42:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 -- start start 2024-09-13T17:42:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340a6020 con 0x7f7d34134e70 2024-09-13T17:42:54.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d32ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d32ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38836/0 (socket says 172.21.15.96:38836) 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d32ffd640 1 -- 172.21.15.96:0/4194670293 learned_addr learned my addr 172.21.15.96:0/4194670293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d32ffd640 1 -- 172.21.15.96:0/4194670293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d240094d0 con 0x7f7d34134e70 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d32ffd640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7d240190f0 tx=0x7f7d2401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d240b7550 con 0x7f7d34134e70 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d240b76b0 con 0x7f7d34134e70 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d340a1460 con 0x7f7d34134e70 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7d240b7990 con 0x7f7d34134e70 2024-09-13T17:42:54.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3413b480 con 0x7f7d34134e70 2024-09-13T17:42:54.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3412fa60 con 0x7f7d34134e70 2024-09-13T17:42:54.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f7d240b3040 con 0x7f7d34134e70 2024-09-13T17:42:54.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 0x7f7d20037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f7d240e1cf0 con 0x7f7d34134e70 2024-09-13T17:42:54.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d327fc640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 0x7f7d20037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.223+0000 7f7d327fc640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 0x7f7d20037910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d28001380 tx=0x7f7d280077d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.231+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7d240c8940 con 0x7f7d34134e70 2024-09-13T17:42:54.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.407+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d3409f5d0 con 0x7f7d34134e70 2024-09-13T17:42:54.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.407+0000 7f7d1b7fe640 1 -- 172.21.15.96:0/4194670293 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7f7d240b1030 con 0x7f7d34134e70 2024-09-13T17:42:54.412 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:54.412 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:42:54.412 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:42:54.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 msgr2=0x7f7d20037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 0x7f7d20037910 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7d28001380 tx=0x7f7d280077d0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 msgr2=0x7f7d340a1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7d240190f0 tx=0x7f7d2401ad20 comp rx=0 tx=0).stop 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 shutdown_connections 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f7d20035450 0x7f7d20037910 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 --2- 172.21.15.96:0/4194670293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d34134e70 0x7f7d340a1be0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 >> 172.21.15.96:0/4194670293 conn(0x7f7d34098a60 msgr2=0x7f7d34133010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 shutdown_connections 2024-09-13T17:42:54.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.411+0000 7f7d394ca640 1 -- 172.21.15.96:0/4194670293 wait complete. 2024-09-13T17:42:54.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:42:54.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd35d80a640 1 Processor -- start 2024-09-13T17:42:54.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd35d80a640 1 -- start start 2024-09-13T17:42:54.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd35d80a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd35806f6d0 0x7fd358134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd35d80a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd35806e130 con 0x7fd35809dc20 2024-09-13T17:42:54.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd35d80a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3580a6750 con 0x7fd35806f6d0 2024-09-13T17:42:54.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd356ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd35806f6d0 0x7fd358134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd3567fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd35809dc20 0x7fd358134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44332/0 (socket says 172.21.15.96:44332) 2024-09-13T17:42:54.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.543+0000 7fd3567fc640 1 -- 172.21.15.96:0/291817105 learned_addr learned my addr 172.21.15.96:0/291817105 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:54.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd356ffd640 1 -- 172.21.15.96:0/291817105 >> v1:172.21.15.96:6789/0 conn(0x7fd35809dc20 legacy=0x7fd358134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd356ffd640 1 -- 172.21.15.96:0/291817105 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580712e0 con 0x7fd35806f6d0 2024-09-13T17:42:54.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd355ffb640 1 -- 172.21.15.96:0/291817105 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783666042 0 0) 0x7fd35806e130 con 0x7fd35809dc20 2024-09-13T17:42:54.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd356ffd640 1 --2- 172.21.15.96:0/291817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358134ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009740 tx=0x7fd34c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35cac46e4f538c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd355ffb640 1 -- 172.21.15.96:0/291817105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34c0ab400 con 0x7fd35806f6d0 2024-09-13T17:42:54.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd355ffb640 1 -- 172.21.15.96:0/291817105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0a7050 con 0x7fd35806f6d0 2024-09-13T17:42:54.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd355ffb640 1 -- 172.21.15.96:0/291817105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34c0ab6b0 con 0x7fd35806f6d0 2024-09-13T17:42:54.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 -- 172.21.15.96:0/291817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 msgr2=0x7fd358134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 --2- 172.21.15.96:0/291817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358134ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd34c009740 tx=0x7fd34c01c390 comp rx=0 tx=0).stop 2024-09-13T17:42:54.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 -- 172.21.15.96:0/291817105 shutdown_connections 2024-09-13T17:42:54.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 --2- 172.21.15.96:0/291817105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358134ab0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 -- 172.21.15.96:0/291817105 >> 172.21.15.96:0/291817105 conn(0x7fd358098a60 msgr2=0x7fd35809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 -- 172.21.15.96:0/291817105 shutdown_connections 2024-09-13T17:42:54.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.547+0000 7fd35d80a640 1 -- 172.21.15.96:0/291817105 wait complete. 2024-09-13T17:42:54.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 Processor -- start 2024-09-13T17:42:54.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 -- start start 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580a3f80 con 0x7fd35806f6d0 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd356ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38858/0 (socket says 172.21.15.96:38858) 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd356ffd640 1 -- 172.21.15.96:0/240335441 learned_addr learned my addr 172.21.15.96:0/240335441 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:54.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd356ffd640 1 -- 172.21.15.96:0/240335441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c009420 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd356ffd640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd34c019fc0 tx=0x7fd34c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34c0abea0 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c004020 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3580a4fd0 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd34c0042c0 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3580a5200 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fd34c0b2050 con 0x7fd35806f6d0 2024-09-13T17:42:54.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd358134ab0 con 0x7fd35806f6d0 2024-09-13T17:42:54.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 0x7fd344037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fd34c0e19e0 con 0x7fd35806f6d0 2024-09-13T17:42:54.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd3567fc640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 0x7fd344037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.551+0000 7fd3567fc640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 0x7fd344037910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd348007fd0 tx=0x7fd348007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.559+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd34c0b2300 con 0x7fd35806f6d0 2024-09-13T17:42:54.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd35806a1b0 con 0x7fd35806f6d0 2024-09-13T17:42:54.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd33ffff640 1 -- 172.21.15.96:0/240335441 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fd34c0b0030 con 0x7fd35806f6d0 2024-09-13T17:42:54.719 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:54.719 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:42:52.060172+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 msgr2=0x7fd344037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 0x7fd344037910 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd348007fd0 tx=0x7fd348007920 comp rx=0 tx=0).stop 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 msgr2=0x7fd358098ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd34c019fc0 tx=0x7fd34c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 shutdown_connections 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fd344035450 0x7fd344037910 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 --2- 172.21.15.96:0/240335441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd35806f6d0 0x7fd358098ec0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 >> 172.21.15.96:0/240335441 conn(0x7fd358098a60 msgr2=0x7fd35812fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 shutdown_connections 2024-09-13T17:42:54.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.715+0000 7fd35d80a640 1 -- 172.21.15.96:0/240335441 wait complete. 2024-09-13T17:42:54.736 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:42:54.736 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:54.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f1080180640 1 Processor -- start 2024-09-13T17:42:54.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f1080180640 1 -- start start 2024-09-13T17:42:54.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f1080180640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f107813af50 0x7f107813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f1080180640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1078138050 con 0x7f107809d690 2024-09-13T17:42:54.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f1080180640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107809c410 con 0x7f107813af50 2024-09-13T17:42:54.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f107d71b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f107813af50 0x7f107813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.851+0000 7f107df1c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f107809d690 0x7f1078138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44340/0 (socket says 172.21.15.96:44340) 2024-09-13T17:42:54.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107df1c640 1 -- 172.21.15.96:0/2104334811 learned_addr learned my addr 172.21.15.96:0/2104334811 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:54.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107d71b640 1 -- 172.21.15.96:0/2104334811 >> v1:172.21.15.96:6789/0 conn(0x7f107809d690 legacy=0x7f1078138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107d71b640 1 -- 172.21.15.96:0/2104334811 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107809cba0 con 0x7f107813af50 2024-09-13T17:42:54.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107d71b640 1 --2- 172.21.15.96:0/2104334811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107813af50 0x7f107813d340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1068009740 tx=0x7f106801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dfae67f4e85a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107cf1a640 1 -- 172.21.15.96:0/2104334811 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10680a8840 con 0x7f107813af50 2024-09-13T17:42:54.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107cf1a640 1 -- 172.21.15.96:0/2104334811 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10680a89a0 con 0x7f107813af50 2024-09-13T17:42:54.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f107cf1a640 1 -- 172.21.15.96:0/2104334811 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10680a8c60 con 0x7f107813af50 2024-09-13T17:42:54.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 -- 172.21.15.96:0/2104334811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107813af50 msgr2=0x7f107813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:54.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 --2- 172.21.15.96:0/2104334811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107813af50 0x7f107813d340 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1068009740 tx=0x7f106801ec10 comp rx=0 tx=0).stop 2024-09-13T17:42:54.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 -- 172.21.15.96:0/2104334811 shutdown_connections 2024-09-13T17:42:54.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 --2- 172.21.15.96:0/2104334811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107813af50 0x7f107813d340 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:54.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 -- 172.21.15.96:0/2104334811 >> 172.21.15.96:0/2104334811 conn(0x7f107806bc60 msgr2=0x7f107806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:54.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 -- 172.21.15.96:0/2104334811 shutdown_connections 2024-09-13T17:42:54.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.855+0000 7f1080180640 1 -- 172.21.15.96:0/2104334811 wait complete. 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 Processor -- start 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 -- start start 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10780a2d40 con 0x7f107809d690 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107df1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107df1c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38880/0 (socket says 172.21.15.96:38880) 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107df1c640 1 -- 172.21.15.96:0/1154915676 learned_addr learned my addr 172.21.15.96:0/1154915676 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107df1c640 1 -- 172.21.15.96:0/1154915676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009420 con 0x7f107809d690 2024-09-13T17:42:54.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107df1c640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f107400c380 tx=0x7f107400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1074005c20 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1074005d80 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10781372e0 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1074007200 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10780a5100 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10780a4f10 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f1074007360 con 0x7f107809d690 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 0x7f105c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:54.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f1074046990 con 0x7f107809d690 2024-09-13T17:42:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107d71b640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 0x7f105c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:54.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.859+0000 7f107d71b640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 0x7f105c037910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10680b2000 tx=0x7f10680aa070 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:54.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:54.863+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1074007610 con 0x7f107809d690 2024-09-13T17:42:55.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.007+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f107809b260 con 0x7f105c035450 2024-09-13T17:42:55.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.015+0000 7f106e7fc640 1 -- 172.21.15.96:0/1154915676 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+244448 (secure 0 0 0) 0x7f10680a8c90 con 0x7f105c035450 2024-09-13T17:42:55.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:55.022 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:55.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 msgr2=0x7f105c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 0x7f105c037910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f10680b2000 tx=0x7f10680aa070 comp rx=0 tx=0).stop 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 msgr2=0x7f10781c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f107400c380 tx=0x7f107400c840 comp rx=0 tx=0).stop 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 shutdown_connections 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f105c035450 0x7f105c037910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 --2- 172.21.15.96:0/1154915676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107809d690 0x7f10781c9550 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 >> 172.21.15.96:0/1154915676 conn(0x7f107806bc60 msgr2=0x7f1078131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 shutdown_connections 2024-09-13T17:42:55.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.019+0000 7f1080180640 1 -- 172.21.15.96:0/1154915676 wait complete. 2024-09-13T17:42:55.037 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-13T17:42:53.808505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82808,"kb_used_data":1744,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032712,"statfs":{"total":289910292480,"available":289825497088,"internally_reserved":0,"allocated":1785856,"data_stored":1374667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.141185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697844+0000","last_change":"2024-09-13T17:42:50.697844+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697844+0000","last_undegraded":"2024-09-13T17:42:50.697844+0000","last_fullsized":"2024-09-13T17:42:50.697844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708801+0000","last_change":"2024-09-13T17:42:50.708801+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708801+0000","last_undegraded":"2024-09-13T17:42:50.708801+0000","last_fullsized":"2024-09-13T17:42:50.708801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.710847+0000","last_change":"2024-09-13T17:42:50.710847+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.710847+0000","last_undegraded":"2024-09-13T17:42:50.710847+0000","last_fullsized":"2024-09-13T17:42:50.710847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698091+0000","last_change":"2024-09-13T17:42:50.698091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698091+0000","last_undegraded":"2024-09-13T17:42:50.698091+0000","last_fullsized":"2024-09-13T17:42:50.698091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708343+0000","last_change":"2024-09-13T17:42:50.708343+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708343+0000","last_undegraded":"2024-09-13T17:42:50.708343+0000","last_fullsized":"2024-09-13T17:42:50.708343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698504+0000","last_change":"2024-09-13T17:42:50.698504+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698504+0000","last_undegraded":"2024-09-13T17:42:50.698504+0000","last_fullsized":"2024-09-13T17:42:50.698504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697639+0000","last_change":"2024-09-13T17:42:50.697639+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697639+0000","last_undegraded":"2024-09-13T17:42:50.697639+0000","last_fullsized":"2024-09-13T17:42:50.697639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708360+0000","last_change":"2024-09-13T17:42:50.708360+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708360+0000","last_undegraded":"2024-09-13T17:42:50.708360+0000","last_fullsized":"2024-09-13T17:42:50.708360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.707967+0000","last_change":"2024-09-13T17:42:50.707967+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.707967+0000","last_undegraded":"2024-09-13T17:42:50.707967+0000","last_fullsized":"2024-09-13T17:42:50.707967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705349+0000","last_change":"2024-09-13T17:42:50.705349+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705349+0000","last_undegraded":"2024-09-13T17:42:50.705349+0000","last_fullsized":"2024-09-13T17:42:50.705349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708384+0000","last_change":"2024-09-13T17:42:50.708384+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708384+0000","last_undegraded":"2024-09-13T17:42:50.708384+0000","last_fullsized":"2024-09-13T17:42:50.708384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696230+0000","last_change":"2024-09-13T17:42:50.696230+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696230+0000","last_undegraded":"2024-09-13T17:42:50.696230+0000","last_fullsized":"2024-09-13T17:42:50.696230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705522+0000","last_change":"2024-09-13T17:42:50.705522+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705522+0000","last_undegraded":"2024-09-13T17:42:50.705522+0000","last_fullsized":"2024-09-13T17:42:50.705522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699632+0000","last_change":"2024-09-13T17:42:50.699632+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699632+0000","last_undegraded":"2024-09-13T17:42:50.699632+0000","last_fullsized":"2024-09-13T17:42:50.699632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699190+0000","last_change":"2024-09-13T17:42:50.699190+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699190+0000","last_undegraded":"2024-09-13T17:42:50.699190+0000","last_fullsized":"2024-09-13T17:42:50.699190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699984+0000","last_change":"2024-09-13T17:42:50.699984+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699984+0000","last_undegraded":"2024-09-13T17:42:50.699984+0000","last_fullsized":"2024-09-13T17:42:50.699984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699637+0000","last_change":"2024-09-13T17:42:50.699637+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699637+0000","last_undegraded":"2024-09-13T17:42:50.699637+0000","last_fullsized":"2024-09-13T17:42:50.699637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695823+0000","last_change":"2024-09-13T17:42:50.695823+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695823+0000","last_undegraded":"2024-09-13T17:42:50.695823+0000","last_fullsized":"2024-09-13T17:42:50.695823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697278+0000","last_change":"2024-09-13T17:42:50.697278+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697278+0000","last_undegraded":"2024-09-13T17:42:50.697278+0000","last_fullsized":"2024-09-13T17:42:50.697278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700418+0000","last_change":"2024-09-13T17:42:50.700418+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700418+0000","last_undegraded":"2024-09-13T17:42:50.700418+0000","last_fullsized":"2024-09-13T17:42:50.700418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696790+0000","last_change":"2024-09-13T17:42:50.696790+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696790+0000","last_undegraded":"2024-09-13T17:42:50.696790+0000","last_fullsized":"2024-09-13T17:42:50.696790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695279+0000","last_change":"2024-09-13T17:42:50.695279+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695279+0000","last_undegraded":"2024-09-13T17:42:50.695279+0000","last_fullsized":"2024-09-13T17:42:50.695279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697222+0000","last_change":"2024-09-13T17:42:50.697222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697222+0000","last_undegraded":"2024-09-13T17:42:50.697222+0000","last_fullsized":"2024-09-13T17:42:50.697222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705812+0000","last_change":"2024-09-13T17:42:50.705812+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705812+0000","last_undegraded":"2024-09-13T17:42:50.705812+0000","last_fullsized":"2024-09-13T17:42:50.705812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.704785+0000","last_change":"2024-09-13T17:42:50.704785+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.704785+0000","last_undegraded":"2024-09-13T17:42:50.704785+0000","last_fullsized":"2024-09-13T17:42:50.704785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.701412+0000","last_change":"2024-09-13T17:42:50.701412+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.701412+0000","last_undegraded":"2024-09-13T17:42:50.701412+0000","last_fullsized":"2024-09-13T17:42:50.701412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696233+0000","last_change":"2024-09-13T17:42:50.696233+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696233+0000","last_undegraded":"2024-09-13T17:42:50.696233+0000","last_fullsized":"2024-09-13T17:42:50.696233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695759+0000","last_change":"2024-09-13T17:42:50.695759+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695759+0000","last_undegraded":"2024-09-13T17:42:50.695759+0000","last_fullsized":"2024-09-13T17:42:50.695759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694091+0000","last_change":"2024-09-13T17:42:50.694091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694091+0000","last_undegraded":"2024-09-13T17:42:50.694091+0000","last_fullsized":"2024-09-13T17:42:50.694091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696667+0000","last_change":"2024-09-13T17:42:50.696667+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696667+0000","last_undegraded":"2024-09-13T17:42:50.696667+0000","last_fullsized":"2024-09-13T17:42:50.696667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696514+0000","last_change":"2024-09-13T17:42:50.696514+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696514+0000","last_undegraded":"2024-09-13T17:42:50.696514+0000","last_fullsized":"2024-09-13T17:42:50.696514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694989+0000","last_change":"2024-09-13T17:42:50.694989+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694989+0000","last_undegraded":"2024-09-13T17:42:50.694989+0000","last_fullsized":"2024-09-13T17:42:50.694989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731524+0000","last_change":"2024-09-13T17:42:50.729720+0000","last_active":"2024-09-13T17:42:50.731524+0000","last_peered":"2024-09-13T17:42:50.731524+0000","last_clean":"2024-09-13T17:42:50.731524+0000","last_became_active":"2024-09-13T17:42:50.729403+0000","last_became_peered":"2024-09-13T17:42:50.729403+0000","last_unstale":"2024-09-13T17:42:50.731524+0000","last_undegraded":"2024-09-13T17:42:50.731524+0000","last_fullsized":"2024-09-13T17:42:50.731524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:02.933863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696149+0000","last_change":"2024-09-13T17:42:50.696149+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696149+0000","last_undegraded":"2024-09-13T17:42:50.696149+0000","last_fullsized":"2024-09-13T17:42:50.696149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731455+0000","last_change":"2024-09-13T17:42:50.729650+0000","last_active":"2024-09-13T17:42:50.731455+0000","last_peered":"2024-09-13T17:42:50.731455+0000","last_clean":"2024-09-13T17:42:50.731455+0000","last_became_active":"2024-09-13T17:42:50.729126+0000","last_became_peered":"2024-09-13T17:42:50.729126+0000","last_unstale":"2024-09-13T17:42:50.731455+0000","last_undegraded":"2024-09-13T17:42:50.731455+0000","last_fullsized":"2024-09-13T17:42:50.731455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:09.600109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694460+0000","last_change":"2024-09-13T17:42:50.694460+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694460+0000","last_undegraded":"2024-09-13T17:42:50.694460+0000","last_fullsized":"2024-09-13T17:42:50.694460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728468+0000","last_change":"2024-09-13T17:42:50.727195+0000","last_active":"2024-09-13T17:42:50.728468+0000","last_peered":"2024-09-13T17:42:50.728468+0000","last_clean":"2024-09-13T17:42:50.728468+0000","last_became_active":"2024-09-13T17:42:50.726846+0000","last_became_peered":"2024-09-13T17:42:50.726846+0000","last_unstale":"2024-09-13T17:42:50.728468+0000","last_undegraded":"2024-09-13T17:42:50.728468+0000","last_fullsized":"2024-09-13T17:42:50.728468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:11.620151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732055+0000","last_change":"2024-09-13T17:42:50.730528+0000","last_active":"2024-09-13T17:42:50.732055+0000","last_peered":"2024-09-13T17:42:50.732055+0000","last_clean":"2024-09-13T17:42:50.732055+0000","last_became_active":"2024-09-13T17:42:50.730190+0000","last_became_peered":"2024-09-13T17:42:50.730190+0000","last_unstale":"2024-09-13T17:42:50.732055+0000","last_undegraded":"2024-09-13T17:42:50.732055+0000","last_fullsized":"2024-09-13T17:42:50.732055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693222+0000","last_change":"2024-09-13T17:42:50.693222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693222+0000","last_undegraded":"2024-09-13T17:42:50.693222+0000","last_fullsized":"2024-09-13T17:42:50.693222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731453+0000","last_change":"2024-09-13T17:42:50.729206+0000","last_active":"2024-09-13T17:42:50.731453+0000","last_peered":"2024-09-13T17:42:50.731453+0000","last_clean":"2024-09-13T17:42:50.731453+0000","last_became_active":"2024-09-13T17:42:50.728865+0000","last_became_peered":"2024-09-13T17:42:50.728865+0000","last_unstale":"2024-09-13T17:42:50.731453+0000","last_undegraded":"2024-09-13T17:42:50.731453+0000","last_fullsized":"2024-09-13T17:42:50.731453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:09.981862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695070+0000","last_change":"2024-09-13T17:42:50.695070+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695070+0000","last_undegraded":"2024-09-13T17:42:50.695070+0000","last_fullsized":"2024-09-13T17:42:50.695070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.725764+0000","last_change":"2024-09-13T17:42:50.719752+0000","last_active":"2024-09-13T17:42:50.725764+0000","last_peered":"2024-09-13T17:42:50.725764+0000","last_clean":"2024-09-13T17:42:50.725764+0000","last_became_active":"2024-09-13T17:42:50.719113+0000","last_became_peered":"2024-09-13T17:42:50.719113+0000","last_unstale":"2024-09-13T17:42:50.725764+0000","last_undegraded":"2024-09-13T17:42:50.725764+0000","last_fullsized":"2024-09-13T17:42:50.725764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:14.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693187+0000","last_change":"2024-09-13T17:42:50.693187+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693187+0000","last_undegraded":"2024-09-13T17:42:50.693187+0000","last_fullsized":"2024-09-13T17:42:50.693187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717741+0000","last_change":"2024-09-13T17:42:50.711236+0000","last_active":"2024-09-13T17:42:50.717741+0000","last_peered":"2024-09-13T17:42:50.717741+0000","last_clean":"2024-09-13T17:42:50.717741+0000","last_became_active":"2024-09-13T17:42:50.710880+0000","last_became_peered":"2024-09-13T17:42:50.710880+0000","last_unstale":"2024-09-13T17:42:50.717741+0000","last_undegraded":"2024-09-13T17:42:50.717741+0000","last_fullsized":"2024-09-13T17:42:50.717741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.473381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693746+0000","last_change":"2024-09-13T17:42:50.693746+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693746+0000","last_undegraded":"2024-09-13T17:42:50.693746+0000","last_fullsized":"2024-09-13T17:42:50.693746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732006+0000","last_change":"2024-09-13T17:42:50.729497+0000","last_active":"2024-09-13T17:42:50.732006+0000","last_peered":"2024-09-13T17:42:50.732006+0000","last_clean":"2024-09-13T17:42:50.732006+0000","last_became_active":"2024-09-13T17:42:50.728723+0000","last_became_peered":"2024-09-13T17:42:50.728723+0000","last_unstale":"2024-09-13T17:42:50.732006+0000","last_undegraded":"2024-09-13T17:42:50.732006+0000","last_fullsized":"2024-09-13T17:42:50.732006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:36.241145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.690548+0000","last_change":"2024-09-13T17:42:50.690548+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.690548+0000","last_undegraded":"2024-09-13T17:42:50.690548+0000","last_fullsized":"2024-09-13T17:42:50.690548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731541+0000","last_change":"2024-09-13T17:42:50.729826+0000","last_active":"2024-09-13T17:42:50.731541+0000","last_peered":"2024-09-13T17:42:50.731541+0000","last_clean":"2024-09-13T17:42:50.731541+0000","last_became_active":"2024-09-13T17:42:50.729515+0000","last_became_peered":"2024-09-13T17:42:50.729515+0000","last_unstale":"2024-09-13T17:42:50.731541+0000","last_undegraded":"2024-09-13T17:42:50.731541+0000","last_fullsized":"2024-09-13T17:42:50.731541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:18.016511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694137+0000","last_change":"2024-09-13T17:42:50.694137+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694137+0000","last_undegraded":"2024-09-13T17:42:50.694137+0000","last_fullsized":"2024-09-13T17:42:50.694137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717647+0000","last_change":"2024-09-13T17:42:50.714076+0000","last_active":"2024-09-13T17:42:50.717647+0000","last_peered":"2024-09-13T17:42:50.717647+0000","last_clean":"2024-09-13T17:42:50.717647+0000","last_became_active":"2024-09-13T17:42:50.713459+0000","last_became_peered":"2024-09-13T17:42:50.713459+0000","last_unstale":"2024-09-13T17:42:50.717647+0000","last_undegraded":"2024-09-13T17:42:50.717647+0000","last_fullsized":"2024-09-13T17:42:50.717647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:23.522612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693969+0000","last_change":"2024-09-13T17:42:50.693969+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693969+0000","last_undegraded":"2024-09-13T17:42:50.693969+0000","last_fullsized":"2024-09-13T17:42:50.693969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.730807+0000","last_change":"2024-09-13T17:42:50.729261+0000","last_active":"2024-09-13T17:42:50.730807+0000","last_peered":"2024-09-13T17:42:50.730807+0000","last_clean":"2024-09-13T17:42:50.730807+0000","last_became_active":"2024-09-13T17:42:50.728991+0000","last_became_peered":"2024-09-13T17:42:50.728991+0000","last_unstale":"2024-09-13T17:42:50.730807+0000","last_undegraded":"2024-09-13T17:42:50.730807+0000","last_fullsized":"2024-09-13T17:42:50.730807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:43.188314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694860+0000","last_change":"2024-09-13T17:42:50.694860+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694860+0000","last_undegraded":"2024-09-13T17:42:50.694860+0000","last_fullsized":"2024-09-13T17:42:50.694860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728549+0000","last_change":"2024-09-13T17:42:50.711610+0000","last_active":"2024-09-13T17:42:50.728549+0000","last_peered":"2024-09-13T17:42:50.728549+0000","last_clean":"2024-09-13T17:42:50.728549+0000","last_became_active":"2024-09-13T17:42:50.711348+0000","last_became_peered":"2024-09-13T17:42:50.711348+0000","last_unstale":"2024-09-13T17:42:50.728549+0000","last_undegraded":"2024-09-13T17:42:50.728549+0000","last_fullsized":"2024-09-13T17:42:50.728549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:41.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692348+0000","last_change":"2024-09-13T17:42:50.692348+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692348+0000","last_undegraded":"2024-09-13T17:42:50.692348+0000","last_fullsized":"2024-09-13T17:42:50.692348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731476+0000","last_change":"2024-09-13T17:42:50.729302+0000","last_active":"2024-09-13T17:42:50.731476+0000","last_peered":"2024-09-13T17:42:50.731476+0000","last_clean":"2024-09-13T17:42:50.731476+0000","last_became_active":"2024-09-13T17:42:50.728824+0000","last_became_peered":"2024-09-13T17:42:50.728824+0000","last_unstale":"2024-09-13T17:42:50.731476+0000","last_undegraded":"2024-09-13T17:42:50.731476+0000","last_fullsized":"2024-09-13T17:42:50.731476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:15:18.318656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693342+0000","last_change":"2024-09-13T17:42:50.693342+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693342+0000","last_undegraded":"2024-09-13T17:42:50.693342+0000","last_fullsized":"2024-09-13T17:42:50.693342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729236+0000","last_change":"2024-09-13T17:42:50.719066+0000","last_active":"2024-09-13T17:42:50.729236+0000","last_peered":"2024-09-13T17:42:50.729236+0000","last_clean":"2024-09-13T17:42:50.729236+0000","last_became_active":"2024-09-13T17:42:50.717841+0000","last_became_peered":"2024-09-13T17:42:50.717841+0000","last_unstale":"2024-09-13T17:42:50.729236+0000","last_undegraded":"2024-09-13T17:42:50.729236+0000","last_fullsized":"2024-09-13T17:42:50.729236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692083+0000","last_change":"2024-09-13T17:42:50.692083+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692083+0000","last_undegraded":"2024-09-13T17:42:50.692083+0000","last_fullsized":"2024-09-13T17:42:50.692083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.721315+0000","last_change":"2024-09-13T17:42:50.715306+0000","last_active":"2024-09-13T17:42:50.721315+0000","last_peered":"2024-09-13T17:42:50.721315+0000","last_clean":"2024-09-13T17:42:50.721315+0000","last_became_active":"2024-09-13T17:42:50.714696+0000","last_became_peered":"2024-09-13T17:42:50.714696+0000","last_unstale":"2024-09-13T17:42:50.721315+0000","last_undegraded":"2024-09-13T17:42:50.721315+0000","last_fullsized":"2024-09-13T17:42:50.721315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693304+0000","last_change":"2024-09-13T17:42:50.693304+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693304+0000","last_undegraded":"2024-09-13T17:42:50.693304+0000","last_fullsized":"2024-09-13T17:42:50.693304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728412+0000","last_change":"2024-09-13T17:42:50.707367+0000","last_active":"2024-09-13T17:42:50.728412+0000","last_peered":"2024-09-13T17:42:50.728412+0000","last_clean":"2024-09-13T17:42:50.728412+0000","last_became_active":"2024-09-13T17:42:50.706061+0000","last_became_peered":"2024-09-13T17:42:50.706061+0000","last_unstale":"2024-09-13T17:42:50.728412+0000","last_undegraded":"2024-09-13T17:42:50.728412+0000","last_fullsized":"2024-09-13T17:42:50.728412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:05.864527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696713+0000","last_change":"2024-09-13T17:42:50.696713+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696713+0000","last_undegraded":"2024-09-13T17:42:50.696713+0000","last_fullsized":"2024-09-13T17:42:50.696713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728606+0000","last_change":"2024-09-13T17:42:50.712051+0000","last_active":"2024-09-13T17:42:50.728606+0000","last_peered":"2024-09-13T17:42:50.728606+0000","last_clean":"2024-09-13T17:42:50.728606+0000","last_became_active":"2024-09-13T17:42:50.711326+0000","last_became_peered":"2024-09-13T17:42:50.711326+0000","last_unstale":"2024-09-13T17:42:50.728606+0000","last_undegraded":"2024-09-13T17:42:50.728606+0000","last_fullsized":"2024-09-13T17:42:50.728606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693033+0000","last_change":"2024-09-13T17:42:50.693033+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693033+0000","last_undegraded":"2024-09-13T17:42:50.693033+0000","last_fullsized":"2024-09-13T17:42:50.693033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732048+0000","last_change":"2024-09-13T17:42:50.730529+0000","last_active":"2024-09-13T17:42:50.732048+0000","last_peered":"2024-09-13T17:42:50.732048+0000","last_clean":"2024-09-13T17:42:50.732048+0000","last_became_active":"2024-09-13T17:42:50.730251+0000","last_became_peered":"2024-09-13T17:42:50.730251+0000","last_unstale":"2024-09-13T17:42:50.732048+0000","last_undegraded":"2024-09-13T17:42:50.732048+0000","last_fullsized":"2024-09-13T17:42:50.732048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:31.396994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687292+0000","last_change":"2024-09-13T17:42:50.687292+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687292+0000","last_undegraded":"2024-09-13T17:42:50.687292+0000","last_fullsized":"2024-09-13T17:42:50.687292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729338+0000","last_change":"2024-09-13T17:42:50.716986+0000","last_active":"2024-09-13T17:42:50.729338+0000","last_peered":"2024-09-13T17:42:50.729338+0000","last_clean":"2024-09-13T17:42:50.729338+0000","last_became_active":"2024-09-13T17:42:50.716657+0000","last_became_peered":"2024-09-13T17:42:50.716657+0000","last_unstale":"2024-09-13T17:42:50.729338+0000","last_undegraded":"2024-09-13T17:42:50.729338+0000","last_fullsized":"2024-09-13T17:42:50.729338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:16.071866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696247+0000","last_change":"2024-09-13T17:42:50.696247+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696247+0000","last_undegraded":"2024-09-13T17:42:50.696247+0000","last_fullsized":"2024-09-13T17:42:50.696247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.719978+0000","last_change":"2024-09-13T17:42:50.716335+0000","last_active":"2024-09-13T17:42:50.719978+0000","last_peered":"2024-09-13T17:42:50.719978+0000","last_clean":"2024-09-13T17:42:50.719978+0000","last_became_active":"2024-09-13T17:42:50.716044+0000","last_became_peered":"2024-09-13T17:42:50.716044+0000","last_unstale":"2024-09-13T17:42:50.719978+0000","last_undegraded":"2024-09-13T17:42:50.719978+0000","last_fullsized":"2024-09-13T17:42:50.719978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:50.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691952+0000","last_change":"2024-09-13T17:42:50.691952+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691952+0000","last_undegraded":"2024-09-13T17:42:50.691952+0000","last_fullsized":"2024-09-13T17:42:50.691952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.723702+0000","last_change":"2024-09-13T17:42:50.717268+0000","last_active":"2024-09-13T17:42:50.723702+0000","last_peered":"2024-09-13T17:42:50.723702+0000","last_clean":"2024-09-13T17:42:50.723702+0000","last_became_active":"2024-09-13T17:42:50.716630+0000","last_became_peered":"2024-09-13T17:42:50.716630+0000","last_unstale":"2024-09-13T17:42:50.723702+0000","last_undegraded":"2024-09-13T17:42:50.723702+0000","last_fullsized":"2024-09-13T17:42:50.723702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:56.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687270+0000","last_change":"2024-09-13T17:42:50.687270+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687270+0000","last_undegraded":"2024-09-13T17:42:50.687270+0000","last_fullsized":"2024-09-13T17:42:50.687270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731410+0000","last_change":"2024-09-13T17:42:50.729119+0000","last_active":"2024-09-13T17:42:50.731410+0000","last_peered":"2024-09-13T17:42:50.731410+0000","last_clean":"2024-09-13T17:42:50.731410+0000","last_became_active":"2024-09-13T17:42:50.728779+0000","last_became_peered":"2024-09-13T17:42:50.728779+0000","last_unstale":"2024-09-13T17:42:50.731410+0000","last_undegraded":"2024-09-13T17:42:50.731410+0000","last_fullsized":"2024-09-13T17:42:50.731410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691690+0000","last_change":"2024-09-13T17:42:50.691690+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691690+0000","last_undegraded":"2024-09-13T17:42:50.691690+0000","last_fullsized":"2024-09-13T17:42:50.691690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729125+0000","last_change":"2024-09-13T17:42:50.714804+0000","last_active":"2024-09-13T17:42:50.729125+0000","last_peered":"2024-09-13T17:42:50.729125+0000","last_clean":"2024-09-13T17:42:50.729125+0000","last_became_active":"2024-09-13T17:42:50.711927+0000","last_became_peered":"2024-09-13T17:42:50.711927+0000","last_unstale":"2024-09-13T17:42:50.729125+0000","last_undegraded":"2024-09-13T17:42:50.729125+0000","last_fullsized":"2024-09-13T17:42:50.729125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:04.749440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689421+0000","last_change":"2024-09-13T17:42:50.689421+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689421+0000","last_undegraded":"2024-09-13T17:42:50.689421+0000","last_fullsized":"2024-09-13T17:42:50.689421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729364+0000","last_change":"2024-09-13T17:42:50.716142+0000","last_active":"2024-09-13T17:42:50.729364+0000","last_peered":"2024-09-13T17:42:50.729364+0000","last_clean":"2024-09-13T17:42:50.729364+0000","last_became_active":"2024-09-13T17:42:50.715549+0000","last_became_peered":"2024-09-13T17:42:50.715549+0000","last_unstale":"2024-09-13T17:42:50.729364+0000","last_undegraded":"2024-09-13T17:42:50.729364+0000","last_fullsized":"2024-09-13T17:42:50.729364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691550+0000","last_change":"2024-09-13T17:42:50.691550+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691550+0000","last_undegraded":"2024-09-13T17:42:50.691550+0000","last_fullsized":"2024-09-13T17:42:50.691550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728587+0000","last_change":"2024-09-13T17:42:50.707353+0000","last_active":"2024-09-13T17:42:50.728587+0000","last_peered":"2024-09-13T17:42:50.728587+0000","last_clean":"2024-09-13T17:42:50.728587+0000","last_became_active":"2024-09-13T17:42:50.706452+0000","last_became_peered":"2024-09-13T17:42:50.706452+0000","last_unstale":"2024-09-13T17:42:50.728587+0000","last_undegraded":"2024-09-13T17:42:50.728587+0000","last_fullsized":"2024-09-13T17:42:50.728587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:15.294714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694956+0000","last_change":"2024-09-13T17:42:50.694956+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694956+0000","last_undegraded":"2024-09-13T17:42:50.694956+0000","last_fullsized":"2024-09-13T17:42:50.694956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731520+0000","last_change":"2024-09-13T17:42:50.730051+0000","last_active":"2024-09-13T17:42:50.731520+0000","last_peered":"2024-09-13T17:42:50.731520+0000","last_clean":"2024-09-13T17:42:50.731520+0000","last_became_active":"2024-09-13T17:42:50.729726+0000","last_became_peered":"2024-09-13T17:42:50.729726+0000","last_unstale":"2024-09-13T17:42:50.731520+0000","last_undegraded":"2024-09-13T17:42:50.731520+0000","last_fullsized":"2024-09-13T17:42:50.731520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:20.073192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691599+0000","last_change":"2024-09-13T17:42:50.691599+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691599+0000","last_undegraded":"2024-09-13T17:42:50.691599+0000","last_fullsized":"2024-09-13T17:42:50.691599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729188+0000","last_change":"2024-09-13T17:42:50.717327+0000","last_active":"2024-09-13T17:42:50.729188+0000","last_peered":"2024-09-13T17:42:50.729188+0000","last_clean":"2024-09-13T17:42:50.729188+0000","last_became_active":"2024-09-13T17:42:50.716762+0000","last_became_peered":"2024-09-13T17:42:50.716762+0000","last_unstale":"2024-09-13T17:42:50.729188+0000","last_undegraded":"2024-09-13T17:42:50.729188+0000","last_fullsized":"2024-09-13T17:42:50.729188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:17.303267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691810+0000","last_change":"2024-09-13T17:42:50.691810+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691810+0000","last_undegraded":"2024-09-13T17:42:50.691810+0000","last_fullsized":"2024-09-13T17:42:50.691810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717516+0000","last_change":"2024-09-13T17:42:50.711721+0000","last_active":"2024-09-13T17:42:50.717516+0000","last_peered":"2024-09-13T17:42:50.717516+0000","last_clean":"2024-09-13T17:42:50.717516+0000","last_became_active":"2024-09-13T17:42:50.711072+0000","last_became_peered":"2024-09-13T17:42:50.711072+0000","last_unstale":"2024-09-13T17:42:50.717516+0000","last_undegraded":"2024-09-13T17:42:50.717516+0000","last_fullsized":"2024-09-13T17:42:50.717516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:53.463804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693573+0000","last_change":"2024-09-13T17:42:50.693573+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693573+0000","last_undegraded":"2024-09-13T17:42:50.693573+0000","last_fullsized":"2024-09-13T17:42:50.693573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729218+0000","last_change":"2024-09-13T17:42:50.718435+0000","last_active":"2024-09-13T17:42:50.729218+0000","last_peered":"2024-09-13T17:42:50.729218+0000","last_clean":"2024-09-13T17:42:50.729218+0000","last_became_active":"2024-09-13T17:42:50.712419+0000","last_became_peered":"2024-09-13T17:42:50.712419+0000","last_unstale":"2024-09-13T17:42:50.729218+0000","last_undegraded":"2024-09-13T17:42:50.729218+0000","last_fullsized":"2024-09-13T17:42:50.729218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:34.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691245+0000","last_change":"2024-09-13T17:42:50.691245+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691245+0000","last_undegraded":"2024-09-13T17:42:50.691245+0000","last_fullsized":"2024-09-13T17:42:50.691245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.726179+0000","last_change":"2024-09-13T17:42:50.724850+0000","last_active":"2024-09-13T17:42:50.726179+0000","last_peered":"2024-09-13T17:42:50.726179+0000","last_clean":"2024-09-13T17:42:50.726179+0000","last_became_active":"2024-09-13T17:42:50.724216+0000","last_became_peered":"2024-09-13T17:42:50.724216+0000","last_unstale":"2024-09-13T17:42:50.726179+0000","last_undegraded":"2024-09-13T17:42:50.726179+0000","last_fullsized":"2024-09-13T17:42:50.726179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689035+0000","last_change":"2024-09-13T17:42:50.689035+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689035+0000","last_undegraded":"2024-09-13T17:42:50.689035+0000","last_fullsized":"2024-09-13T17:42:50.689035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.722805+0000","last_change":"2024-09-13T17:42:50.717580+0000","last_active":"2024-09-13T17:42:50.722805+0000","last_peered":"2024-09-13T17:42:50.722805+0000","last_clean":"2024-09-13T17:42:50.722805+0000","last_became_active":"2024-09-13T17:42:50.716737+0000","last_became_peered":"2024-09-13T17:42:50.716737+0000","last_unstale":"2024-09-13T17:42:50.722805+0000","last_undegraded":"2024-09-13T17:42:50.722805+0000","last_fullsized":"2024-09-13T17:42:50.722805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:44.217628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691862+0000","last_change":"2024-09-13T17:42:50.691862+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691862+0000","last_undegraded":"2024-09-13T17:42:50.691862+0000","last_fullsized":"2024-09-13T17:42:50.691862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729058+0000","last_change":"2024-09-13T17:42:50.712393+0000","last_active":"2024-09-13T17:42:50.729058+0000","last_peered":"2024-09-13T17:42:50.729058+0000","last_clean":"2024-09-13T17:42:50.729058+0000","last_became_active":"2024-09-13T17:42:50.712046+0000","last_became_peered":"2024-09-13T17:42:50.712046+0000","last_unstale":"2024-09-13T17:42:50.729058+0000","last_undegraded":"2024-09-13T17:42:50.729058+0000","last_fullsized":"2024-09-13T17:42:50.729058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:49.680242+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:49.680242+0000","last_peered":"2024-09-13T17:42:49.680242+0000","last_clean":"2024-09-13T17:42:49.680242+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:49.680242+0000","last_undegraded":"2024-09-13T17:42:49.680242+0000","last_fullsized":"2024-09-13T17:42:49.680242+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738383,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":63322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":653690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:55.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:55.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626884b640 1 Processor -- start 2024-09-13T17:42:55.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626884b640 1 -- start start 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626884b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f626406f6d0 0x7f6264134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626884b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f626406e130 con 0x7f626409dc20 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626884b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640a6750 con 0x7f626406f6d0 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f626406f6d0 0x7f6264134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f625bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f626409dc20 0x7f6264134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44342/0 (socket says 172.21.15.96:44342) 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f626406f6d0 0x7f6264134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38886/0 (socket says 172.21.15.96:38886) 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f625bfff640 1 -- 172.21.15.96:0/2923582994 learned_addr learned my addr 172.21.15.96:0/2923582994 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626259c640 1 -- 172.21.15.96:0/2923582994 >> v1:172.21.15.96:6789/0 conn(0x7f626409dc20 legacy=0x7f6264134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626259c640 1 -- 172.21.15.96:0/2923582994 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640712e0 con 0x7f626406f6d0 2024-09-13T17:42:55.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f6261d9b640 1 -- 172.21.15.96:0/2923582994 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328466500 0 0) 0x7f626406e130 con 0x7f626409dc20 2024-09-13T17:42:55.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f626259c640 1 --2- 172.21.15.96:0/2923582994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f6264134ab0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6254009740 tx=0x7f625401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5569b091c69da7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f6261d9b640 1 -- 172.21.15.96:0/2923582994 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62540ab400 con 0x7f626406f6d0 2024-09-13T17:42:55.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f6261d9b640 1 -- 172.21.15.96:0/2923582994 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540a7050 con 0x7f626406f6d0 2024-09-13T17:42:55.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.191+0000 7f6261d9b640 1 -- 172.21.15.96:0/2923582994 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62540ab6b0 con 0x7f626406f6d0 2024-09-13T17:42:55.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/2923582994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 msgr2=0x7f6264134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 --2- 172.21.15.96:0/2923582994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f6264134ab0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f6254009740 tx=0x7f625401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/2923582994 shutdown_connections 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 --2- 172.21.15.96:0/2923582994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f6264134ab0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/2923582994 >> 172.21.15.96:0/2923582994 conn(0x7f6264098a60 msgr2=0x7f626409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/2923582994 shutdown_connections 2024-09-13T17:42:55.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/2923582994 wait complete. 2024-09-13T17:42:55.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 Processor -- start 2024-09-13T17:42:55.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- start start 2024-09-13T17:42:55.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640a5b50 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38900/0 (socket says 172.21.15.96:38900) 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626259c640 1 -- 172.21.15.96:0/1782065822 learned_addr learned my addr 172.21.15.96:0/1782065822 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626259c640 1 -- 172.21.15.96:0/1782065822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009420 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626259c640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6254019fc0 tx=0x7f625401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62540b7450 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62540b75b0 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62640a4440 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62540b7850 con 0x7f626406f6d0 2024-09-13T17:42:55.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640a4670 con 0x7f626406f6d0 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626409a360 con 0x7f626406f6d0 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f62540b79b0 con 0x7f626406f6d0 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f62540e20d0 con 0x7f626406f6d0 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625bfff640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.195+0000 7f625bfff640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f624c007fd0 tx=0x7f624c007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.199+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f62540c76c0 con 0x7f626406f6d0 2024-09-13T17:42:55.320 INFO:tasks.ceph.mds.b.smithi096.stderr:2024-09-13T17:42:55.315+0000 7f1754c82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:42:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.343+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6264135910 con 0x7f6244035450 2024-09-13T17:42:55.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.347+0000 7f625affd640 1 -- 172.21.15.96:0/1782065822 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+244448 (secure 0 0 0) 0x7f624c0070d0 con 0x7f6244035450 2024-09-13T17:42:55.354 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:55.356 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 msgr2=0x7f6244037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f624c007fd0 tx=0x7f624c007920 comp rx=0 tx=0).stop 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 msgr2=0x7f62640a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6254019fc0 tx=0x7f625401abd0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 shutdown_connections 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f6244035450 0x7f6244037910 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 --2- 172.21.15.96:0/1782065822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f626406f6d0 0x7f62640a5610 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 >> 172.21.15.96:0/1782065822 conn(0x7f6264098a60 msgr2=0x7f62641368d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 shutdown_connections 2024-09-13T17:42:55.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.351+0000 7f626884b640 1 -- 172.21.15.96:0/1782065822 wait complete. 2024-09-13T17:42:55.371 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-13T17:42:53.808505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82808,"kb_used_data":1744,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032712,"statfs":{"total":289910292480,"available":289825497088,"internally_reserved":0,"allocated":1785856,"data_stored":1374667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.141185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697844+0000","last_change":"2024-09-13T17:42:50.697844+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697844+0000","last_undegraded":"2024-09-13T17:42:50.697844+0000","last_fullsized":"2024-09-13T17:42:50.697844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708801+0000","last_change":"2024-09-13T17:42:50.708801+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708801+0000","last_undegraded":"2024-09-13T17:42:50.708801+0000","last_fullsized":"2024-09-13T17:42:50.708801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.710847+0000","last_change":"2024-09-13T17:42:50.710847+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.710847+0000","last_undegraded":"2024-09-13T17:42:50.710847+0000","last_fullsized":"2024-09-13T17:42:50.710847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698091+0000","last_change":"2024-09-13T17:42:50.698091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698091+0000","last_undegraded":"2024-09-13T17:42:50.698091+0000","last_fullsized":"2024-09-13T17:42:50.698091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708343+0000","last_change":"2024-09-13T17:42:50.708343+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708343+0000","last_undegraded":"2024-09-13T17:42:50.708343+0000","last_fullsized":"2024-09-13T17:42:50.708343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698504+0000","last_change":"2024-09-13T17:42:50.698504+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698504+0000","last_undegraded":"2024-09-13T17:42:50.698504+0000","last_fullsized":"2024-09-13T17:42:50.698504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697639+0000","last_change":"2024-09-13T17:42:50.697639+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697639+0000","last_undegraded":"2024-09-13T17:42:50.697639+0000","last_fullsized":"2024-09-13T17:42:50.697639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708360+0000","last_change":"2024-09-13T17:42:50.708360+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708360+0000","last_undegraded":"2024-09-13T17:42:50.708360+0000","last_fullsized":"2024-09-13T17:42:50.708360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.707967+0000","last_change":"2024-09-13T17:42:50.707967+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.707967+0000","last_undegraded":"2024-09-13T17:42:50.707967+0000","last_fullsized":"2024-09-13T17:42:50.707967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705349+0000","last_change":"2024-09-13T17:42:50.705349+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705349+0000","last_undegraded":"2024-09-13T17:42:50.705349+0000","last_fullsized":"2024-09-13T17:42:50.705349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708384+0000","last_change":"2024-09-13T17:42:50.708384+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708384+0000","last_undegraded":"2024-09-13T17:42:50.708384+0000","last_fullsized":"2024-09-13T17:42:50.708384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696230+0000","last_change":"2024-09-13T17:42:50.696230+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696230+0000","last_undegraded":"2024-09-13T17:42:50.696230+0000","last_fullsized":"2024-09-13T17:42:50.696230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705522+0000","last_change":"2024-09-13T17:42:50.705522+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705522+0000","last_undegraded":"2024-09-13T17:42:50.705522+0000","last_fullsized":"2024-09-13T17:42:50.705522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699632+0000","last_change":"2024-09-13T17:42:50.699632+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699632+0000","last_undegraded":"2024-09-13T17:42:50.699632+0000","last_fullsized":"2024-09-13T17:42:50.699632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699190+0000","last_change":"2024-09-13T17:42:50.699190+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699190+0000","last_undegraded":"2024-09-13T17:42:50.699190+0000","last_fullsized":"2024-09-13T17:42:50.699190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699984+0000","last_change":"2024-09-13T17:42:50.699984+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699984+0000","last_undegraded":"2024-09-13T17:42:50.699984+0000","last_fullsized":"2024-09-13T17:42:50.699984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699637+0000","last_change":"2024-09-13T17:42:50.699637+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699637+0000","last_undegraded":"2024-09-13T17:42:50.699637+0000","last_fullsized":"2024-09-13T17:42:50.699637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695823+0000","last_change":"2024-09-13T17:42:50.695823+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695823+0000","last_undegraded":"2024-09-13T17:42:50.695823+0000","last_fullsized":"2024-09-13T17:42:50.695823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697278+0000","last_change":"2024-09-13T17:42:50.697278+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697278+0000","last_undegraded":"2024-09-13T17:42:50.697278+0000","last_fullsized":"2024-09-13T17:42:50.697278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700418+0000","last_change":"2024-09-13T17:42:50.700418+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700418+0000","last_undegraded":"2024-09-13T17:42:50.700418+0000","last_fullsized":"2024-09-13T17:42:50.700418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696790+0000","last_change":"2024-09-13T17:42:50.696790+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696790+0000","last_undegraded":"2024-09-13T17:42:50.696790+0000","last_fullsized":"2024-09-13T17:42:50.696790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695279+0000","last_change":"2024-09-13T17:42:50.695279+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695279+0000","last_undegraded":"2024-09-13T17:42:50.695279+0000","last_fullsized":"2024-09-13T17:42:50.695279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697222+0000","last_change":"2024-09-13T17:42:50.697222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697222+0000","last_undegraded":"2024-09-13T17:42:50.697222+0000","last_fullsized":"2024-09-13T17:42:50.697222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705812+0000","last_change":"2024-09-13T17:42:50.705812+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705812+0000","last_undegraded":"2024-09-13T17:42:50.705812+0000","last_fullsized":"2024-09-13T17:42:50.705812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.704785+0000","last_change":"2024-09-13T17:42:50.704785+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.704785+0000","last_undegraded":"2024-09-13T17:42:50.704785+0000","last_fullsized":"2024-09-13T17:42:50.704785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.701412+0000","last_change":"2024-09-13T17:42:50.701412+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.701412+0000","last_undegraded":"2024-09-13T17:42:50.701412+0000","last_fullsized":"2024-09-13T17:42:50.701412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696233+0000","last_change":"2024-09-13T17:42:50.696233+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696233+0000","last_undegraded":"2024-09-13T17:42:50.696233+0000","last_fullsized":"2024-09-13T17:42:50.696233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695759+0000","last_change":"2024-09-13T17:42:50.695759+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695759+0000","last_undegraded":"2024-09-13T17:42:50.695759+0000","last_fullsized":"2024-09-13T17:42:50.695759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694091+0000","last_change":"2024-09-13T17:42:50.694091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694091+0000","last_undegraded":"2024-09-13T17:42:50.694091+0000","last_fullsized":"2024-09-13T17:42:50.694091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696667+0000","last_change":"2024-09-13T17:42:50.696667+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696667+0000","last_undegraded":"2024-09-13T17:42:50.696667+0000","last_fullsized":"2024-09-13T17:42:50.696667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696514+0000","last_change":"2024-09-13T17:42:50.696514+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696514+0000","last_undegraded":"2024-09-13T17:42:50.696514+0000","last_fullsized":"2024-09-13T17:42:50.696514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694989+0000","last_change":"2024-09-13T17:42:50.694989+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694989+0000","last_undegraded":"2024-09-13T17:42:50.694989+0000","last_fullsized":"2024-09-13T17:42:50.694989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731524+0000","last_change":"2024-09-13T17:42:50.729720+0000","last_active":"2024-09-13T17:42:50.731524+0000","last_peered":"2024-09-13T17:42:50.731524+0000","last_clean":"2024-09-13T17:42:50.731524+0000","last_became_active":"2024-09-13T17:42:50.729403+0000","last_became_peered":"2024-09-13T17:42:50.729403+0000","last_unstale":"2024-09-13T17:42:50.731524+0000","last_undegraded":"2024-09-13T17:42:50.731524+0000","last_fullsized":"2024-09-13T17:42:50.731524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:02.933863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696149+0000","last_change":"2024-09-13T17:42:50.696149+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696149+0000","last_undegraded":"2024-09-13T17:42:50.696149+0000","last_fullsized":"2024-09-13T17:42:50.696149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731455+0000","last_change":"2024-09-13T17:42:50.729650+0000","last_active":"2024-09-13T17:42:50.731455+0000","last_peered":"2024-09-13T17:42:50.731455+0000","last_clean":"2024-09-13T17:42:50.731455+0000","last_became_active":"2024-09-13T17:42:50.729126+0000","last_became_peered":"2024-09-13T17:42:50.729126+0000","last_unstale":"2024-09-13T17:42:50.731455+0000","last_undegraded":"2024-09-13T17:42:50.731455+0000","last_fullsized":"2024-09-13T17:42:50.731455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:09.600109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694460+0000","last_change":"2024-09-13T17:42:50.694460+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694460+0000","last_undegraded":"2024-09-13T17:42:50.694460+0000","last_fullsized":"2024-09-13T17:42:50.694460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728468+0000","last_change":"2024-09-13T17:42:50.727195+0000","last_active":"2024-09-13T17:42:50.728468+0000","last_peered":"2024-09-13T17:42:50.728468+0000","last_clean":"2024-09-13T17:42:50.728468+0000","last_became_active":"2024-09-13T17:42:50.726846+0000","last_became_peered":"2024-09-13T17:42:50.726846+0000","last_unstale":"2024-09-13T17:42:50.728468+0000","last_undegraded":"2024-09-13T17:42:50.728468+0000","last_fullsized":"2024-09-13T17:42:50.728468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:11.620151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732055+0000","last_change":"2024-09-13T17:42:50.730528+0000","last_active":"2024-09-13T17:42:50.732055+0000","last_peered":"2024-09-13T17:42:50.732055+0000","last_clean":"2024-09-13T17:42:50.732055+0000","last_became_active":"2024-09-13T17:42:50.730190+0000","last_became_peered":"2024-09-13T17:42:50.730190+0000","last_unstale":"2024-09-13T17:42:50.732055+0000","last_undegraded":"2024-09-13T17:42:50.732055+0000","last_fullsized":"2024-09-13T17:42:50.732055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693222+0000","last_change":"2024-09-13T17:42:50.693222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693222+0000","last_undegraded":"2024-09-13T17:42:50.693222+0000","last_fullsized":"2024-09-13T17:42:50.693222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731453+0000","last_change":"2024-09-13T17:42:50.729206+0000","last_active":"2024-09-13T17:42:50.731453+0000","last_peered":"2024-09-13T17:42:50.731453+0000","last_clean":"2024-09-13T17:42:50.731453+0000","last_became_active":"2024-09-13T17:42:50.728865+0000","last_became_peered":"2024-09-13T17:42:50.728865+0000","last_unstale":"2024-09-13T17:42:50.731453+0000","last_undegraded":"2024-09-13T17:42:50.731453+0000","last_fullsized":"2024-09-13T17:42:50.731453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:09.981862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695070+0000","last_change":"2024-09-13T17:42:50.695070+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695070+0000","last_undegraded":"2024-09-13T17:42:50.695070+0000","last_fullsized":"2024-09-13T17:42:50.695070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.725764+0000","last_change":"2024-09-13T17:42:50.719752+0000","last_active":"2024-09-13T17:42:50.725764+0000","last_peered":"2024-09-13T17:42:50.725764+0000","last_clean":"2024-09-13T17:42:50.725764+0000","last_became_active":"2024-09-13T17:42:50.719113+0000","last_became_peered":"2024-09-13T17:42:50.719113+0000","last_unstale":"2024-09-13T17:42:50.725764+0000","last_undegraded":"2024-09-13T17:42:50.725764+0000","last_fullsized":"2024-09-13T17:42:50.725764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:14.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693187+0000","last_change":"2024-09-13T17:42:50.693187+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693187+0000","last_undegraded":"2024-09-13T17:42:50.693187+0000","last_fullsized":"2024-09-13T17:42:50.693187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717741+0000","last_change":"2024-09-13T17:42:50.711236+0000","last_active":"2024-09-13T17:42:50.717741+0000","last_peered":"2024-09-13T17:42:50.717741+0000","last_clean":"2024-09-13T17:42:50.717741+0000","last_became_active":"2024-09-13T17:42:50.710880+0000","last_became_peered":"2024-09-13T17:42:50.710880+0000","last_unstale":"2024-09-13T17:42:50.717741+0000","last_undegraded":"2024-09-13T17:42:50.717741+0000","last_fullsized":"2024-09-13T17:42:50.717741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.473381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693746+0000","last_change":"2024-09-13T17:42:50.693746+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693746+0000","last_undegraded":"2024-09-13T17:42:50.693746+0000","last_fullsized":"2024-09-13T17:42:50.693746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732006+0000","last_change":"2024-09-13T17:42:50.729497+0000","last_active":"2024-09-13T17:42:50.732006+0000","last_peered":"2024-09-13T17:42:50.732006+0000","last_clean":"2024-09-13T17:42:50.732006+0000","last_became_active":"2024-09-13T17:42:50.728723+0000","last_became_peered":"2024-09-13T17:42:50.728723+0000","last_unstale":"2024-09-13T17:42:50.732006+0000","last_undegraded":"2024-09-13T17:42:50.732006+0000","last_fullsized":"2024-09-13T17:42:50.732006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:36.241145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.690548+0000","last_change":"2024-09-13T17:42:50.690548+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.690548+0000","last_undegraded":"2024-09-13T17:42:50.690548+0000","last_fullsized":"2024-09-13T17:42:50.690548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731541+0000","last_change":"2024-09-13T17:42:50.729826+0000","last_active":"2024-09-13T17:42:50.731541+0000","last_peered":"2024-09-13T17:42:50.731541+0000","last_clean":"2024-09-13T17:42:50.731541+0000","last_became_active":"2024-09-13T17:42:50.729515+0000","last_became_peered":"2024-09-13T17:42:50.729515+0000","last_unstale":"2024-09-13T17:42:50.731541+0000","last_undegraded":"2024-09-13T17:42:50.731541+0000","last_fullsized":"2024-09-13T17:42:50.731541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:18.016511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694137+0000","last_change":"2024-09-13T17:42:50.694137+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694137+0000","last_undegraded":"2024-09-13T17:42:50.694137+0000","last_fullsized":"2024-09-13T17:42:50.694137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717647+0000","last_change":"2024-09-13T17:42:50.714076+0000","last_active":"2024-09-13T17:42:50.717647+0000","last_peered":"2024-09-13T17:42:50.717647+0000","last_clean":"2024-09-13T17:42:50.717647+0000","last_became_active":"2024-09-13T17:42:50.713459+0000","last_became_peered":"2024-09-13T17:42:50.713459+0000","last_unstale":"2024-09-13T17:42:50.717647+0000","last_undegraded":"2024-09-13T17:42:50.717647+0000","last_fullsized":"2024-09-13T17:42:50.717647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:23.522612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693969+0000","last_change":"2024-09-13T17:42:50.693969+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693969+0000","last_undegraded":"2024-09-13T17:42:50.693969+0000","last_fullsized":"2024-09-13T17:42:50.693969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.730807+0000","last_change":"2024-09-13T17:42:50.729261+0000","last_active":"2024-09-13T17:42:50.730807+0000","last_peered":"2024-09-13T17:42:50.730807+0000","last_clean":"2024-09-13T17:42:50.730807+0000","last_became_active":"2024-09-13T17:42:50.728991+0000","last_became_peered":"2024-09-13T17:42:50.728991+0000","last_unstale":"2024-09-13T17:42:50.730807+0000","last_undegraded":"2024-09-13T17:42:50.730807+0000","last_fullsized":"2024-09-13T17:42:50.730807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:43.188314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694860+0000","last_change":"2024-09-13T17:42:50.694860+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694860+0000","last_undegraded":"2024-09-13T17:42:50.694860+0000","last_fullsized":"2024-09-13T17:42:50.694860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728549+0000","last_change":"2024-09-13T17:42:50.711610+0000","last_active":"2024-09-13T17:42:50.728549+0000","last_peered":"2024-09-13T17:42:50.728549+0000","last_clean":"2024-09-13T17:42:50.728549+0000","last_became_active":"2024-09-13T17:42:50.711348+0000","last_became_peered":"2024-09-13T17:42:50.711348+0000","last_unstale":"2024-09-13T17:42:50.728549+0000","last_undegraded":"2024-09-13T17:42:50.728549+0000","last_fullsized":"2024-09-13T17:42:50.728549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:41.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692348+0000","last_change":"2024-09-13T17:42:50.692348+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692348+0000","last_undegraded":"2024-09-13T17:42:50.692348+0000","last_fullsized":"2024-09-13T17:42:50.692348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731476+0000","last_change":"2024-09-13T17:42:50.729302+0000","last_active":"2024-09-13T17:42:50.731476+0000","last_peered":"2024-09-13T17:42:50.731476+0000","last_clean":"2024-09-13T17:42:50.731476+0000","last_became_active":"2024-09-13T17:42:50.728824+0000","last_became_peered":"2024-09-13T17:42:50.728824+0000","last_unstale":"2024-09-13T17:42:50.731476+0000","last_undegraded":"2024-09-13T17:42:50.731476+0000","last_fullsized":"2024-09-13T17:42:50.731476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:15:18.318656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693342+0000","last_change":"2024-09-13T17:42:50.693342+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693342+0000","last_undegraded":"2024-09-13T17:42:50.693342+0000","last_fullsized":"2024-09-13T17:42:50.693342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729236+0000","last_change":"2024-09-13T17:42:50.719066+0000","last_active":"2024-09-13T17:42:50.729236+0000","last_peered":"2024-09-13T17:42:50.729236+0000","last_clean":"2024-09-13T17:42:50.729236+0000","last_became_active":"2024-09-13T17:42:50.717841+0000","last_became_peered":"2024-09-13T17:42:50.717841+0000","last_unstale":"2024-09-13T17:42:50.729236+0000","last_undegraded":"2024-09-13T17:42:50.729236+0000","last_fullsized":"2024-09-13T17:42:50.729236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692083+0000","last_change":"2024-09-13T17:42:50.692083+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692083+0000","last_undegraded":"2024-09-13T17:42:50.692083+0000","last_fullsized":"2024-09-13T17:42:50.692083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.721315+0000","last_change":"2024-09-13T17:42:50.715306+0000","last_active":"2024-09-13T17:42:50.721315+0000","last_peered":"2024-09-13T17:42:50.721315+0000","last_clean":"2024-09-13T17:42:50.721315+0000","last_became_active":"2024-09-13T17:42:50.714696+0000","last_became_peered":"2024-09-13T17:42:50.714696+0000","last_unstale":"2024-09-13T17:42:50.721315+0000","last_undegraded":"2024-09-13T17:42:50.721315+0000","last_fullsized":"2024-09-13T17:42:50.721315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693304+0000","last_change":"2024-09-13T17:42:50.693304+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693304+0000","last_undegraded":"2024-09-13T17:42:50.693304+0000","last_fullsized":"2024-09-13T17:42:50.693304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728412+0000","last_change":"2024-09-13T17:42:50.707367+0000","last_active":"2024-09-13T17:42:50.728412+0000","last_peered":"2024-09-13T17:42:50.728412+0000","last_clean":"2024-09-13T17:42:50.728412+0000","last_became_active":"2024-09-13T17:42:50.706061+0000","last_became_peered":"2024-09-13T17:42:50.706061+0000","last_unstale":"2024-09-13T17:42:50.728412+0000","last_undegraded":"2024-09-13T17:42:50.728412+0000","last_fullsized":"2024-09-13T17:42:50.728412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:05.864527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696713+0000","last_change":"2024-09-13T17:42:50.696713+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696713+0000","last_undegraded":"2024-09-13T17:42:50.696713+0000","last_fullsized":"2024-09-13T17:42:50.696713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728606+0000","last_change":"2024-09-13T17:42:50.712051+0000","last_active":"2024-09-13T17:42:50.728606+0000","last_peered":"2024-09-13T17:42:50.728606+0000","last_clean":"2024-09-13T17:42:50.728606+0000","last_became_active":"2024-09-13T17:42:50.711326+0000","last_became_peered":"2024-09-13T17:42:50.711326+0000","last_unstale":"2024-09-13T17:42:50.728606+0000","last_undegraded":"2024-09-13T17:42:50.728606+0000","last_fullsized":"2024-09-13T17:42:50.728606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693033+0000","last_change":"2024-09-13T17:42:50.693033+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693033+0000","last_undegraded":"2024-09-13T17:42:50.693033+0000","last_fullsized":"2024-09-13T17:42:50.693033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732048+0000","last_change":"2024-09-13T17:42:50.730529+0000","last_active":"2024-09-13T17:42:50.732048+0000","last_peered":"2024-09-13T17:42:50.732048+0000","last_clean":"2024-09-13T17:42:50.732048+0000","last_became_active":"2024-09-13T17:42:50.730251+0000","last_became_peered":"2024-09-13T17:42:50.730251+0000","last_unstale":"2024-09-13T17:42:50.732048+0000","last_undegraded":"2024-09-13T17:42:50.732048+0000","last_fullsized":"2024-09-13T17:42:50.732048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:31.396994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687292+0000","last_change":"2024-09-13T17:42:50.687292+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687292+0000","last_undegraded":"2024-09-13T17:42:50.687292+0000","last_fullsized":"2024-09-13T17:42:50.687292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729338+0000","last_change":"2024-09-13T17:42:50.716986+0000","last_active":"2024-09-13T17:42:50.729338+0000","last_peered":"2024-09-13T17:42:50.729338+0000","last_clean":"2024-09-13T17:42:50.729338+0000","last_became_active":"2024-09-13T17:42:50.716657+0000","last_became_peered":"2024-09-13T17:42:50.716657+0000","last_unstale":"2024-09-13T17:42:50.729338+0000","last_undegraded":"2024-09-13T17:42:50.729338+0000","last_fullsized":"2024-09-13T17:42:50.729338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:16.071866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696247+0000","last_change":"2024-09-13T17:42:50.696247+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696247+0000","last_undegraded":"2024-09-13T17:42:50.696247+0000","last_fullsized":"2024-09-13T17:42:50.696247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.719978+0000","last_change":"2024-09-13T17:42:50.716335+0000","last_active":"2024-09-13T17:42:50.719978+0000","last_peered":"2024-09-13T17:42:50.719978+0000","last_clean":"2024-09-13T17:42:50.719978+0000","last_became_active":"2024-09-13T17:42:50.716044+0000","last_became_peered":"2024-09-13T17:42:50.716044+0000","last_unstale":"2024-09-13T17:42:50.719978+0000","last_undegraded":"2024-09-13T17:42:50.719978+0000","last_fullsized":"2024-09-13T17:42:50.719978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:50.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691952+0000","last_change":"2024-09-13T17:42:50.691952+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691952+0000","last_undegraded":"2024-09-13T17:42:50.691952+0000","last_fullsized":"2024-09-13T17:42:50.691952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.723702+0000","last_change":"2024-09-13T17:42:50.717268+0000","last_active":"2024-09-13T17:42:50.723702+0000","last_peered":"2024-09-13T17:42:50.723702+0000","last_clean":"2024-09-13T17:42:50.723702+0000","last_became_active":"2024-09-13T17:42:50.716630+0000","last_became_peered":"2024-09-13T17:42:50.716630+0000","last_unstale":"2024-09-13T17:42:50.723702+0000","last_undegraded":"2024-09-13T17:42:50.723702+0000","last_fullsized":"2024-09-13T17:42:50.723702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:56.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687270+0000","last_change":"2024-09-13T17:42:50.687270+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687270+0000","last_undegraded":"2024-09-13T17:42:50.687270+0000","last_fullsized":"2024-09-13T17:42:50.687270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731410+0000","last_change":"2024-09-13T17:42:50.729119+0000","last_active":"2024-09-13T17:42:50.731410+0000","last_peered":"2024-09-13T17:42:50.731410+0000","last_clean":"2024-09-13T17:42:50.731410+0000","last_became_active":"2024-09-13T17:42:50.728779+0000","last_became_peered":"2024-09-13T17:42:50.728779+0000","last_unstale":"2024-09-13T17:42:50.731410+0000","last_undegraded":"2024-09-13T17:42:50.731410+0000","last_fullsized":"2024-09-13T17:42:50.731410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691690+0000","last_change":"2024-09-13T17:42:50.691690+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691690+0000","last_undegraded":"2024-09-13T17:42:50.691690+0000","last_fullsized":"2024-09-13T17:42:50.691690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729125+0000","last_change":"2024-09-13T17:42:50.714804+0000","last_active":"2024-09-13T17:42:50.729125+0000","last_peered":"2024-09-13T17:42:50.729125+0000","last_clean":"2024-09-13T17:42:50.729125+0000","last_became_active":"2024-09-13T17:42:50.711927+0000","last_became_peered":"2024-09-13T17:42:50.711927+0000","last_unstale":"2024-09-13T17:42:50.729125+0000","last_undegraded":"2024-09-13T17:42:50.729125+0000","last_fullsized":"2024-09-13T17:42:50.729125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:04.749440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689421+0000","last_change":"2024-09-13T17:42:50.689421+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689421+0000","last_undegraded":"2024-09-13T17:42:50.689421+0000","last_fullsized":"2024-09-13T17:42:50.689421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729364+0000","last_change":"2024-09-13T17:42:50.716142+0000","last_active":"2024-09-13T17:42:50.729364+0000","last_peered":"2024-09-13T17:42:50.729364+0000","last_clean":"2024-09-13T17:42:50.729364+0000","last_became_active":"2024-09-13T17:42:50.715549+0000","last_became_peered":"2024-09-13T17:42:50.715549+0000","last_unstale":"2024-09-13T17:42:50.729364+0000","last_undegraded":"2024-09-13T17:42:50.729364+0000","last_fullsized":"2024-09-13T17:42:50.729364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691550+0000","last_change":"2024-09-13T17:42:50.691550+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691550+0000","last_undegraded":"2024-09-13T17:42:50.691550+0000","last_fullsized":"2024-09-13T17:42:50.691550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728587+0000","last_change":"2024-09-13T17:42:50.707353+0000","last_active":"2024-09-13T17:42:50.728587+0000","last_peered":"2024-09-13T17:42:50.728587+0000","last_clean":"2024-09-13T17:42:50.728587+0000","last_became_active":"2024-09-13T17:42:50.706452+0000","last_became_peered":"2024-09-13T17:42:50.706452+0000","last_unstale":"2024-09-13T17:42:50.728587+0000","last_undegraded":"2024-09-13T17:42:50.728587+0000","last_fullsized":"2024-09-13T17:42:50.728587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:15.294714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694956+0000","last_change":"2024-09-13T17:42:50.694956+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694956+0000","last_undegraded":"2024-09-13T17:42:50.694956+0000","last_fullsized":"2024-09-13T17:42:50.694956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731520+0000","last_change":"2024-09-13T17:42:50.730051+0000","last_active":"2024-09-13T17:42:50.731520+0000","last_peered":"2024-09-13T17:42:50.731520+0000","last_clean":"2024-09-13T17:42:50.731520+0000","last_became_active":"2024-09-13T17:42:50.729726+0000","last_became_peered":"2024-09-13T17:42:50.729726+0000","last_unstale":"2024-09-13T17:42:50.731520+0000","last_undegraded":"2024-09-13T17:42:50.731520+0000","last_fullsized":"2024-09-13T17:42:50.731520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:20.073192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691599+0000","last_change":"2024-09-13T17:42:50.691599+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691599+0000","last_undegraded":"2024-09-13T17:42:50.691599+0000","last_fullsized":"2024-09-13T17:42:50.691599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729188+0000","last_change":"2024-09-13T17:42:50.717327+0000","last_active":"2024-09-13T17:42:50.729188+0000","last_peered":"2024-09-13T17:42:50.729188+0000","last_clean":"2024-09-13T17:42:50.729188+0000","last_became_active":"2024-09-13T17:42:50.716762+0000","last_became_peered":"2024-09-13T17:42:50.716762+0000","last_unstale":"2024-09-13T17:42:50.729188+0000","last_undegraded":"2024-09-13T17:42:50.729188+0000","last_fullsized":"2024-09-13T17:42:50.729188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:17.303267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691810+0000","last_change":"2024-09-13T17:42:50.691810+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691810+0000","last_undegraded":"2024-09-13T17:42:50.691810+0000","last_fullsized":"2024-09-13T17:42:50.691810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717516+0000","last_change":"2024-09-13T17:42:50.711721+0000","last_active":"2024-09-13T17:42:50.717516+0000","last_peered":"2024-09-13T17:42:50.717516+0000","last_clean":"2024-09-13T17:42:50.717516+0000","last_became_active":"2024-09-13T17:42:50.711072+0000","last_became_peered":"2024-09-13T17:42:50.711072+0000","last_unstale":"2024-09-13T17:42:50.717516+0000","last_undegraded":"2024-09-13T17:42:50.717516+0000","last_fullsized":"2024-09-13T17:42:50.717516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:53.463804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693573+0000","last_change":"2024-09-13T17:42:50.693573+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693573+0000","last_undegraded":"2024-09-13T17:42:50.693573+0000","last_fullsized":"2024-09-13T17:42:50.693573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729218+0000","last_change":"2024-09-13T17:42:50.718435+0000","last_active":"2024-09-13T17:42:50.729218+0000","last_peered":"2024-09-13T17:42:50.729218+0000","last_clean":"2024-09-13T17:42:50.729218+0000","last_became_active":"2024-09-13T17:42:50.712419+0000","last_became_peered":"2024-09-13T17:42:50.712419+0000","last_unstale":"2024-09-13T17:42:50.729218+0000","last_undegraded":"2024-09-13T17:42:50.729218+0000","last_fullsized":"2024-09-13T17:42:50.729218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:34.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691245+0000","last_change":"2024-09-13T17:42:50.691245+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691245+0000","last_undegraded":"2024-09-13T17:42:50.691245+0000","last_fullsized":"2024-09-13T17:42:50.691245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.726179+0000","last_change":"2024-09-13T17:42:50.724850+0000","last_active":"2024-09-13T17:42:50.726179+0000","last_peered":"2024-09-13T17:42:50.726179+0000","last_clean":"2024-09-13T17:42:50.726179+0000","last_became_active":"2024-09-13T17:42:50.724216+0000","last_became_peered":"2024-09-13T17:42:50.724216+0000","last_unstale":"2024-09-13T17:42:50.726179+0000","last_undegraded":"2024-09-13T17:42:50.726179+0000","last_fullsized":"2024-09-13T17:42:50.726179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689035+0000","last_change":"2024-09-13T17:42:50.689035+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689035+0000","last_undegraded":"2024-09-13T17:42:50.689035+0000","last_fullsized":"2024-09-13T17:42:50.689035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.722805+0000","last_change":"2024-09-13T17:42:50.717580+0000","last_active":"2024-09-13T17:42:50.722805+0000","last_peered":"2024-09-13T17:42:50.722805+0000","last_clean":"2024-09-13T17:42:50.722805+0000","last_became_active":"2024-09-13T17:42:50.716737+0000","last_became_peered":"2024-09-13T17:42:50.716737+0000","last_unstale":"2024-09-13T17:42:50.722805+0000","last_undegraded":"2024-09-13T17:42:50.722805+0000","last_fullsized":"2024-09-13T17:42:50.722805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:44.217628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691862+0000","last_change":"2024-09-13T17:42:50.691862+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691862+0000","last_undegraded":"2024-09-13T17:42:50.691862+0000","last_fullsized":"2024-09-13T17:42:50.691862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729058+0000","last_change":"2024-09-13T17:42:50.712393+0000","last_active":"2024-09-13T17:42:50.729058+0000","last_peered":"2024-09-13T17:42:50.729058+0000","last_clean":"2024-09-13T17:42:50.729058+0000","last_became_active":"2024-09-13T17:42:50.712046+0000","last_became_peered":"2024-09-13T17:42:50.712046+0000","last_unstale":"2024-09-13T17:42:50.729058+0000","last_undegraded":"2024-09-13T17:42:50.729058+0000","last_fullsized":"2024-09-13T17:42:50.729058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:49.680242+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:49.680242+0000","last_peered":"2024-09-13T17:42:49.680242+0000","last_clean":"2024-09-13T17:42:49.680242+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:49.680242+0000","last_undegraded":"2024-09-13T17:42:49.680242+0000","last_fullsized":"2024-09-13T17:42:49.680242+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738383,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":63322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":653690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:55.375 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:55.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b7021640 1 Processor -- start 2024-09-13T17:42:55.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b7021640 1 -- start start 2024-09-13T17:42:55.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b7021640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f20b009ead0 0x7f20b0139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b7021640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b00a0da0 con 0x7f20b01349f0 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b7021640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00a1040 con 0x7f20b009ead0 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b4dbd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f20b01349f0 0x7f20b0134dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44352/0 (socket says 172.21.15.96:44352) 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20b4dbd640 1 -- 172.21.15.96:0/4273297463 learned_addr learned my addr 172.21.15.96:0/4273297463 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20affff640 1 --2- 172.21.15.96:0/4273297463 >> v2:172.21.15.96:3300/0 conn(0x7f20b009ead0 0x7f20b0139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20affff640 1 -- 172.21.15.96:0/4273297463 >> v1:172.21.15.96:6789/0 conn(0x7f20b01349f0 legacy=0x7f20b0134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20affff640 1 -- 172.21.15.96:0/4273297463 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00a12e0 con 0x7f20b009ead0 2024-09-13T17:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.527+0000 7f20affff640 1 --2- 172.21.15.96:0/4273297463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0139f90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f209c009740 tx=0x7f209c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faf30aa2e8b64faa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20af7fe640 1 -- 172.21.15.96:0/4273297463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f209c0ab530 con 0x7f20b009ead0 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20af7fe640 1 -- 172.21.15.96:0/4273297463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c0a7050 con 0x7f20b009ead0 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20af7fe640 1 -- 172.21.15.96:0/4273297463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f209c0ab7e0 con 0x7f20b009ead0 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/4273297463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 msgr2=0x7f20b0139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 --2- 172.21.15.96:0/4273297463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0139f90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f209c009740 tx=0x7f209c01c390 comp rx=0 tx=0).stop 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/4273297463 shutdown_connections 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 --2- 172.21.15.96:0/4273297463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0139f90 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/4273297463 >> 172.21.15.96:0/4273297463 conn(0x7f20b00a7390 msgr2=0x7f20b00a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/4273297463 shutdown_connections 2024-09-13T17:42:55.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/4273297463 wait complete. 2024-09-13T17:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 Processor -- start 2024-09-13T17:42:55.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- start start 2024-09-13T17:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b009ac80 con 0x7f20b009ead0 2024-09-13T17:42:55.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b4dbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b4dbd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38912/0 (socket says 172.21.15.96:38912) 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b4dbd640 1 -- 172.21.15.96:0/2310898198 learned_addr learned my addr 172.21.15.96:0/2310898198 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b4dbd640 1 -- 172.21.15.96:0/2310898198 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c009420 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b4dbd640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c480 tx=0x7f20a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20a4005c20 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4005d80 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b0099f80 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20a40071a0 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b0098c30 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f20a4007300 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.531+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b009aa50 con 0x7f20b009ead0 2024-09-13T17:42:55.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.535+0000 7f20ad7fa640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 0x7f2094037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.535+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f20a4046b50 con 0x7f20b009ead0 2024-09-13T17:42:55.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.535+0000 7f20affff640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 0x7f2094037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.535+0000 7f20affff640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 0x7f2094037910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f209c0ad000 tx=0x7f209c0a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.539+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f20a4012400 con 0x7f20b009ead0 2024-09-13T17:42:55.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.683+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20b013ccf0 con 0x7f2094035450 2024-09-13T17:42:55.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.687+0000 7f20ad7fa640 1 -- 172.21.15.96:0/2310898198 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+244448 (secure 0 0 0) 0x7f209c0ab7a0 con 0x7f2094035450 2024-09-13T17:42:55.692 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:55.695 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:55.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 msgr2=0x7f2094037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 0x7f2094037910 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f209c0ad000 tx=0x7f209c0a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:55.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 msgr2=0x7f20b0069f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f20a400c480 tx=0x7f20a400c940 comp rx=0 tx=0).stop 2024-09-13T17:42:55.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 shutdown_connections 2024-09-13T17:42:55.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f2094035450 0x7f2094037910 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 --2- 172.21.15.96:0/2310898198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20b009ead0 0x7f20b0069f80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 >> 172.21.15.96:0/2310898198 conn(0x7f20b00a7390 msgr2=0x7f20b00a6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 shutdown_connections 2024-09-13T17:42:55.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.691+0000 7f20b7021640 1 -- 172.21.15.96:0/2310898198 wait complete. 2024-09-13T17:42:55.712 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":55,"stamp":"2024-09-13T17:42:53.808505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82808,"kb_used_data":1744,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032712,"statfs":{"total":289910292480,"available":289825497088,"internally_reserved":0,"allocated":1785856,"data_stored":1374667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.141185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697844+0000","last_change":"2024-09-13T17:42:50.697844+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697844+0000","last_undegraded":"2024-09-13T17:42:50.697844+0000","last_fullsized":"2024-09-13T17:42:50.697844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708801+0000","last_change":"2024-09-13T17:42:50.708801+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708801+0000","last_undegraded":"2024-09-13T17:42:50.708801+0000","last_fullsized":"2024-09-13T17:42:50.708801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.710847+0000","last_change":"2024-09-13T17:42:50.710847+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.710847+0000","last_undegraded":"2024-09-13T17:42:50.710847+0000","last_fullsized":"2024-09-13T17:42:50.710847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698091+0000","last_change":"2024-09-13T17:42:50.698091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698091+0000","last_undegraded":"2024-09-13T17:42:50.698091+0000","last_fullsized":"2024-09-13T17:42:50.698091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708343+0000","last_change":"2024-09-13T17:42:50.708343+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708343+0000","last_undegraded":"2024-09-13T17:42:50.708343+0000","last_fullsized":"2024-09-13T17:42:50.708343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698504+0000","last_change":"2024-09-13T17:42:50.698504+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698504+0000","last_undegraded":"2024-09-13T17:42:50.698504+0000","last_fullsized":"2024-09-13T17:42:50.698504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697639+0000","last_change":"2024-09-13T17:42:50.697639+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697639+0000","last_undegraded":"2024-09-13T17:42:50.697639+0000","last_fullsized":"2024-09-13T17:42:50.697639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708360+0000","last_change":"2024-09-13T17:42:50.708360+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708360+0000","last_undegraded":"2024-09-13T17:42:50.708360+0000","last_fullsized":"2024-09-13T17:42:50.708360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.707967+0000","last_change":"2024-09-13T17:42:50.707967+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.707967+0000","last_undegraded":"2024-09-13T17:42:50.707967+0000","last_fullsized":"2024-09-13T17:42:50.707967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705349+0000","last_change":"2024-09-13T17:42:50.705349+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705349+0000","last_undegraded":"2024-09-13T17:42:50.705349+0000","last_fullsized":"2024-09-13T17:42:50.705349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708384+0000","last_change":"2024-09-13T17:42:50.708384+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708384+0000","last_undegraded":"2024-09-13T17:42:50.708384+0000","last_fullsized":"2024-09-13T17:42:50.708384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696230+0000","last_change":"2024-09-13T17:42:50.696230+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696230+0000","last_undegraded":"2024-09-13T17:42:50.696230+0000","last_fullsized":"2024-09-13T17:42:50.696230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705522+0000","last_change":"2024-09-13T17:42:50.705522+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705522+0000","last_undegraded":"2024-09-13T17:42:50.705522+0000","last_fullsized":"2024-09-13T17:42:50.705522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699632+0000","last_change":"2024-09-13T17:42:50.699632+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699632+0000","last_undegraded":"2024-09-13T17:42:50.699632+0000","last_fullsized":"2024-09-13T17:42:50.699632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699190+0000","last_change":"2024-09-13T17:42:50.699190+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699190+0000","last_undegraded":"2024-09-13T17:42:50.699190+0000","last_fullsized":"2024-09-13T17:42:50.699190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699984+0000","last_change":"2024-09-13T17:42:50.699984+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699984+0000","last_undegraded":"2024-09-13T17:42:50.699984+0000","last_fullsized":"2024-09-13T17:42:50.699984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699637+0000","last_change":"2024-09-13T17:42:50.699637+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699637+0000","last_undegraded":"2024-09-13T17:42:50.699637+0000","last_fullsized":"2024-09-13T17:42:50.699637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695823+0000","last_change":"2024-09-13T17:42:50.695823+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695823+0000","last_undegraded":"2024-09-13T17:42:50.695823+0000","last_fullsized":"2024-09-13T17:42:50.695823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697278+0000","last_change":"2024-09-13T17:42:50.697278+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697278+0000","last_undegraded":"2024-09-13T17:42:50.697278+0000","last_fullsized":"2024-09-13T17:42:50.697278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700418+0000","last_change":"2024-09-13T17:42:50.700418+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700418+0000","last_undegraded":"2024-09-13T17:42:50.700418+0000","last_fullsized":"2024-09-13T17:42:50.700418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696790+0000","last_change":"2024-09-13T17:42:50.696790+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696790+0000","last_undegraded":"2024-09-13T17:42:50.696790+0000","last_fullsized":"2024-09-13T17:42:50.696790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695279+0000","last_change":"2024-09-13T17:42:50.695279+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695279+0000","last_undegraded":"2024-09-13T17:42:50.695279+0000","last_fullsized":"2024-09-13T17:42:50.695279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697222+0000","last_change":"2024-09-13T17:42:50.697222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697222+0000","last_undegraded":"2024-09-13T17:42:50.697222+0000","last_fullsized":"2024-09-13T17:42:50.697222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705812+0000","last_change":"2024-09-13T17:42:50.705812+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705812+0000","last_undegraded":"2024-09-13T17:42:50.705812+0000","last_fullsized":"2024-09-13T17:42:50.705812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.704785+0000","last_change":"2024-09-13T17:42:50.704785+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.704785+0000","last_undegraded":"2024-09-13T17:42:50.704785+0000","last_fullsized":"2024-09-13T17:42:50.704785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.701412+0000","last_change":"2024-09-13T17:42:50.701412+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.701412+0000","last_undegraded":"2024-09-13T17:42:50.701412+0000","last_fullsized":"2024-09-13T17:42:50.701412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696233+0000","last_change":"2024-09-13T17:42:50.696233+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696233+0000","last_undegraded":"2024-09-13T17:42:50.696233+0000","last_fullsized":"2024-09-13T17:42:50.696233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695759+0000","last_change":"2024-09-13T17:42:50.695759+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695759+0000","last_undegraded":"2024-09-13T17:42:50.695759+0000","last_fullsized":"2024-09-13T17:42:50.695759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694091+0000","last_change":"2024-09-13T17:42:50.694091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694091+0000","last_undegraded":"2024-09-13T17:42:50.694091+0000","last_fullsized":"2024-09-13T17:42:50.694091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696667+0000","last_change":"2024-09-13T17:42:50.696667+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696667+0000","last_undegraded":"2024-09-13T17:42:50.696667+0000","last_fullsized":"2024-09-13T17:42:50.696667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696514+0000","last_change":"2024-09-13T17:42:50.696514+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696514+0000","last_undegraded":"2024-09-13T17:42:50.696514+0000","last_fullsized":"2024-09-13T17:42:50.696514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694989+0000","last_change":"2024-09-13T17:42:50.694989+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694989+0000","last_undegraded":"2024-09-13T17:42:50.694989+0000","last_fullsized":"2024-09-13T17:42:50.694989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731524+0000","last_change":"2024-09-13T17:42:50.729720+0000","last_active":"2024-09-13T17:42:50.731524+0000","last_peered":"2024-09-13T17:42:50.731524+0000","last_clean":"2024-09-13T17:42:50.731524+0000","last_became_active":"2024-09-13T17:42:50.729403+0000","last_became_peered":"2024-09-13T17:42:50.729403+0000","last_unstale":"2024-09-13T17:42:50.731524+0000","last_undegraded":"2024-09-13T17:42:50.731524+0000","last_fullsized":"2024-09-13T17:42:50.731524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:02.933863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696149+0000","last_change":"2024-09-13T17:42:50.696149+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696149+0000","last_undegraded":"2024-09-13T17:42:50.696149+0000","last_fullsized":"2024-09-13T17:42:50.696149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731455+0000","last_change":"2024-09-13T17:42:50.729650+0000","last_active":"2024-09-13T17:42:50.731455+0000","last_peered":"2024-09-13T17:42:50.731455+0000","last_clean":"2024-09-13T17:42:50.731455+0000","last_became_active":"2024-09-13T17:42:50.729126+0000","last_became_peered":"2024-09-13T17:42:50.729126+0000","last_unstale":"2024-09-13T17:42:50.731455+0000","last_undegraded":"2024-09-13T17:42:50.731455+0000","last_fullsized":"2024-09-13T17:42:50.731455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:09.600109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694460+0000","last_change":"2024-09-13T17:42:50.694460+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694460+0000","last_undegraded":"2024-09-13T17:42:50.694460+0000","last_fullsized":"2024-09-13T17:42:50.694460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728468+0000","last_change":"2024-09-13T17:42:50.727195+0000","last_active":"2024-09-13T17:42:50.728468+0000","last_peered":"2024-09-13T17:42:50.728468+0000","last_clean":"2024-09-13T17:42:50.728468+0000","last_became_active":"2024-09-13T17:42:50.726846+0000","last_became_peered":"2024-09-13T17:42:50.726846+0000","last_unstale":"2024-09-13T17:42:50.728468+0000","last_undegraded":"2024-09-13T17:42:50.728468+0000","last_fullsized":"2024-09-13T17:42:50.728468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:11.620151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732055+0000","last_change":"2024-09-13T17:42:50.730528+0000","last_active":"2024-09-13T17:42:50.732055+0000","last_peered":"2024-09-13T17:42:50.732055+0000","last_clean":"2024-09-13T17:42:50.732055+0000","last_became_active":"2024-09-13T17:42:50.730190+0000","last_became_peered":"2024-09-13T17:42:50.730190+0000","last_unstale":"2024-09-13T17:42:50.732055+0000","last_undegraded":"2024-09-13T17:42:50.732055+0000","last_fullsized":"2024-09-13T17:42:50.732055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693222+0000","last_change":"2024-09-13T17:42:50.693222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693222+0000","last_undegraded":"2024-09-13T17:42:50.693222+0000","last_fullsized":"2024-09-13T17:42:50.693222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731453+0000","last_change":"2024-09-13T17:42:50.729206+0000","last_active":"2024-09-13T17:42:50.731453+0000","last_peered":"2024-09-13T17:42:50.731453+0000","last_clean":"2024-09-13T17:42:50.731453+0000","last_became_active":"2024-09-13T17:42:50.728865+0000","last_became_peered":"2024-09-13T17:42:50.728865+0000","last_unstale":"2024-09-13T17:42:50.731453+0000","last_undegraded":"2024-09-13T17:42:50.731453+0000","last_fullsized":"2024-09-13T17:42:50.731453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:09.981862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695070+0000","last_change":"2024-09-13T17:42:50.695070+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695070+0000","last_undegraded":"2024-09-13T17:42:50.695070+0000","last_fullsized":"2024-09-13T17:42:50.695070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.725764+0000","last_change":"2024-09-13T17:42:50.719752+0000","last_active":"2024-09-13T17:42:50.725764+0000","last_peered":"2024-09-13T17:42:50.725764+0000","last_clean":"2024-09-13T17:42:50.725764+0000","last_became_active":"2024-09-13T17:42:50.719113+0000","last_became_peered":"2024-09-13T17:42:50.719113+0000","last_unstale":"2024-09-13T17:42:50.725764+0000","last_undegraded":"2024-09-13T17:42:50.725764+0000","last_fullsized":"2024-09-13T17:42:50.725764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:14.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693187+0000","last_change":"2024-09-13T17:42:50.693187+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693187+0000","last_undegraded":"2024-09-13T17:42:50.693187+0000","last_fullsized":"2024-09-13T17:42:50.693187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717741+0000","last_change":"2024-09-13T17:42:50.711236+0000","last_active":"2024-09-13T17:42:50.717741+0000","last_peered":"2024-09-13T17:42:50.717741+0000","last_clean":"2024-09-13T17:42:50.717741+0000","last_became_active":"2024-09-13T17:42:50.710880+0000","last_became_peered":"2024-09-13T17:42:50.710880+0000","last_unstale":"2024-09-13T17:42:50.717741+0000","last_undegraded":"2024-09-13T17:42:50.717741+0000","last_fullsized":"2024-09-13T17:42:50.717741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.473381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693746+0000","last_change":"2024-09-13T17:42:50.693746+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693746+0000","last_undegraded":"2024-09-13T17:42:50.693746+0000","last_fullsized":"2024-09-13T17:42:50.693746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732006+0000","last_change":"2024-09-13T17:42:50.729497+0000","last_active":"2024-09-13T17:42:50.732006+0000","last_peered":"2024-09-13T17:42:50.732006+0000","last_clean":"2024-09-13T17:42:50.732006+0000","last_became_active":"2024-09-13T17:42:50.728723+0000","last_became_peered":"2024-09-13T17:42:50.728723+0000","last_unstale":"2024-09-13T17:42:50.732006+0000","last_undegraded":"2024-09-13T17:42:50.732006+0000","last_fullsized":"2024-09-13T17:42:50.732006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:36.241145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.690548+0000","last_change":"2024-09-13T17:42:50.690548+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.690548+0000","last_undegraded":"2024-09-13T17:42:50.690548+0000","last_fullsized":"2024-09-13T17:42:50.690548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731541+0000","last_change":"2024-09-13T17:42:50.729826+0000","last_active":"2024-09-13T17:42:50.731541+0000","last_peered":"2024-09-13T17:42:50.731541+0000","last_clean":"2024-09-13T17:42:50.731541+0000","last_became_active":"2024-09-13T17:42:50.729515+0000","last_became_peered":"2024-09-13T17:42:50.729515+0000","last_unstale":"2024-09-13T17:42:50.731541+0000","last_undegraded":"2024-09-13T17:42:50.731541+0000","last_fullsized":"2024-09-13T17:42:50.731541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:18.016511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694137+0000","last_change":"2024-09-13T17:42:50.694137+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694137+0000","last_undegraded":"2024-09-13T17:42:50.694137+0000","last_fullsized":"2024-09-13T17:42:50.694137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717647+0000","last_change":"2024-09-13T17:42:50.714076+0000","last_active":"2024-09-13T17:42:50.717647+0000","last_peered":"2024-09-13T17:42:50.717647+0000","last_clean":"2024-09-13T17:42:50.717647+0000","last_became_active":"2024-09-13T17:42:50.713459+0000","last_became_peered":"2024-09-13T17:42:50.713459+0000","last_unstale":"2024-09-13T17:42:50.717647+0000","last_undegraded":"2024-09-13T17:42:50.717647+0000","last_fullsized":"2024-09-13T17:42:50.717647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:23.522612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693969+0000","last_change":"2024-09-13T17:42:50.693969+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693969+0000","last_undegraded":"2024-09-13T17:42:50.693969+0000","last_fullsized":"2024-09-13T17:42:50.693969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.730807+0000","last_change":"2024-09-13T17:42:50.729261+0000","last_active":"2024-09-13T17:42:50.730807+0000","last_peered":"2024-09-13T17:42:50.730807+0000","last_clean":"2024-09-13T17:42:50.730807+0000","last_became_active":"2024-09-13T17:42:50.728991+0000","last_became_peered":"2024-09-13T17:42:50.728991+0000","last_unstale":"2024-09-13T17:42:50.730807+0000","last_undegraded":"2024-09-13T17:42:50.730807+0000","last_fullsized":"2024-09-13T17:42:50.730807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:43.188314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694860+0000","last_change":"2024-09-13T17:42:50.694860+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694860+0000","last_undegraded":"2024-09-13T17:42:50.694860+0000","last_fullsized":"2024-09-13T17:42:50.694860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728549+0000","last_change":"2024-09-13T17:42:50.711610+0000","last_active":"2024-09-13T17:42:50.728549+0000","last_peered":"2024-09-13T17:42:50.728549+0000","last_clean":"2024-09-13T17:42:50.728549+0000","last_became_active":"2024-09-13T17:42:50.711348+0000","last_became_peered":"2024-09-13T17:42:50.711348+0000","last_unstale":"2024-09-13T17:42:50.728549+0000","last_undegraded":"2024-09-13T17:42:50.728549+0000","last_fullsized":"2024-09-13T17:42:50.728549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:41.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692348+0000","last_change":"2024-09-13T17:42:50.692348+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692348+0000","last_undegraded":"2024-09-13T17:42:50.692348+0000","last_fullsized":"2024-09-13T17:42:50.692348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731476+0000","last_change":"2024-09-13T17:42:50.729302+0000","last_active":"2024-09-13T17:42:50.731476+0000","last_peered":"2024-09-13T17:42:50.731476+0000","last_clean":"2024-09-13T17:42:50.731476+0000","last_became_active":"2024-09-13T17:42:50.728824+0000","last_became_peered":"2024-09-13T17:42:50.728824+0000","last_unstale":"2024-09-13T17:42:50.731476+0000","last_undegraded":"2024-09-13T17:42:50.731476+0000","last_fullsized":"2024-09-13T17:42:50.731476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:15:18.318656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693342+0000","last_change":"2024-09-13T17:42:50.693342+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693342+0000","last_undegraded":"2024-09-13T17:42:50.693342+0000","last_fullsized":"2024-09-13T17:42:50.693342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729236+0000","last_change":"2024-09-13T17:42:50.719066+0000","last_active":"2024-09-13T17:42:50.729236+0000","last_peered":"2024-09-13T17:42:50.729236+0000","last_clean":"2024-09-13T17:42:50.729236+0000","last_became_active":"2024-09-13T17:42:50.717841+0000","last_became_peered":"2024-09-13T17:42:50.717841+0000","last_unstale":"2024-09-13T17:42:50.729236+0000","last_undegraded":"2024-09-13T17:42:50.729236+0000","last_fullsized":"2024-09-13T17:42:50.729236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692083+0000","last_change":"2024-09-13T17:42:50.692083+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692083+0000","last_undegraded":"2024-09-13T17:42:50.692083+0000","last_fullsized":"2024-09-13T17:42:50.692083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.721315+0000","last_change":"2024-09-13T17:42:50.715306+0000","last_active":"2024-09-13T17:42:50.721315+0000","last_peered":"2024-09-13T17:42:50.721315+0000","last_clean":"2024-09-13T17:42:50.721315+0000","last_became_active":"2024-09-13T17:42:50.714696+0000","last_became_peered":"2024-09-13T17:42:50.714696+0000","last_unstale":"2024-09-13T17:42:50.721315+0000","last_undegraded":"2024-09-13T17:42:50.721315+0000","last_fullsized":"2024-09-13T17:42:50.721315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693304+0000","last_change":"2024-09-13T17:42:50.693304+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693304+0000","last_undegraded":"2024-09-13T17:42:50.693304+0000","last_fullsized":"2024-09-13T17:42:50.693304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728412+0000","last_change":"2024-09-13T17:42:50.707367+0000","last_active":"2024-09-13T17:42:50.728412+0000","last_peered":"2024-09-13T17:42:50.728412+0000","last_clean":"2024-09-13T17:42:50.728412+0000","last_became_active":"2024-09-13T17:42:50.706061+0000","last_became_peered":"2024-09-13T17:42:50.706061+0000","last_unstale":"2024-09-13T17:42:50.728412+0000","last_undegraded":"2024-09-13T17:42:50.728412+0000","last_fullsized":"2024-09-13T17:42:50.728412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:05.864527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696713+0000","last_change":"2024-09-13T17:42:50.696713+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696713+0000","last_undegraded":"2024-09-13T17:42:50.696713+0000","last_fullsized":"2024-09-13T17:42:50.696713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728606+0000","last_change":"2024-09-13T17:42:50.712051+0000","last_active":"2024-09-13T17:42:50.728606+0000","last_peered":"2024-09-13T17:42:50.728606+0000","last_clean":"2024-09-13T17:42:50.728606+0000","last_became_active":"2024-09-13T17:42:50.711326+0000","last_became_peered":"2024-09-13T17:42:50.711326+0000","last_unstale":"2024-09-13T17:42:50.728606+0000","last_undegraded":"2024-09-13T17:42:50.728606+0000","last_fullsized":"2024-09-13T17:42:50.728606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693033+0000","last_change":"2024-09-13T17:42:50.693033+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693033+0000","last_undegraded":"2024-09-13T17:42:50.693033+0000","last_fullsized":"2024-09-13T17:42:50.693033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732048+0000","last_change":"2024-09-13T17:42:50.730529+0000","last_active":"2024-09-13T17:42:50.732048+0000","last_peered":"2024-09-13T17:42:50.732048+0000","last_clean":"2024-09-13T17:42:50.732048+0000","last_became_active":"2024-09-13T17:42:50.730251+0000","last_became_peered":"2024-09-13T17:42:50.730251+0000","last_unstale":"2024-09-13T17:42:50.732048+0000","last_undegraded":"2024-09-13T17:42:50.732048+0000","last_fullsized":"2024-09-13T17:42:50.732048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:31.396994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687292+0000","last_change":"2024-09-13T17:42:50.687292+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687292+0000","last_undegraded":"2024-09-13T17:42:50.687292+0000","last_fullsized":"2024-09-13T17:42:50.687292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729338+0000","last_change":"2024-09-13T17:42:50.716986+0000","last_active":"2024-09-13T17:42:50.729338+0000","last_peered":"2024-09-13T17:42:50.729338+0000","last_clean":"2024-09-13T17:42:50.729338+0000","last_became_active":"2024-09-13T17:42:50.716657+0000","last_became_peered":"2024-09-13T17:42:50.716657+0000","last_unstale":"2024-09-13T17:42:50.729338+0000","last_undegraded":"2024-09-13T17:42:50.729338+0000","last_fullsized":"2024-09-13T17:42:50.729338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:16.071866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696247+0000","last_change":"2024-09-13T17:42:50.696247+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696247+0000","last_undegraded":"2024-09-13T17:42:50.696247+0000","last_fullsized":"2024-09-13T17:42:50.696247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.719978+0000","last_change":"2024-09-13T17:42:50.716335+0000","last_active":"2024-09-13T17:42:50.719978+0000","last_peered":"2024-09-13T17:42:50.719978+0000","last_clean":"2024-09-13T17:42:50.719978+0000","last_became_active":"2024-09-13T17:42:50.716044+0000","last_became_peered":"2024-09-13T17:42:50.716044+0000","last_unstale":"2024-09-13T17:42:50.719978+0000","last_undegraded":"2024-09-13T17:42:50.719978+0000","last_fullsized":"2024-09-13T17:42:50.719978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:50.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691952+0000","last_change":"2024-09-13T17:42:50.691952+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691952+0000","last_undegraded":"2024-09-13T17:42:50.691952+0000","last_fullsized":"2024-09-13T17:42:50.691952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.723702+0000","last_change":"2024-09-13T17:42:50.717268+0000","last_active":"2024-09-13T17:42:50.723702+0000","last_peered":"2024-09-13T17:42:50.723702+0000","last_clean":"2024-09-13T17:42:50.723702+0000","last_became_active":"2024-09-13T17:42:50.716630+0000","last_became_peered":"2024-09-13T17:42:50.716630+0000","last_unstale":"2024-09-13T17:42:50.723702+0000","last_undegraded":"2024-09-13T17:42:50.723702+0000","last_fullsized":"2024-09-13T17:42:50.723702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:56.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687270+0000","last_change":"2024-09-13T17:42:50.687270+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687270+0000","last_undegraded":"2024-09-13T17:42:50.687270+0000","last_fullsized":"2024-09-13T17:42:50.687270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731410+0000","last_change":"2024-09-13T17:42:50.729119+0000","last_active":"2024-09-13T17:42:50.731410+0000","last_peered":"2024-09-13T17:42:50.731410+0000","last_clean":"2024-09-13T17:42:50.731410+0000","last_became_active":"2024-09-13T17:42:50.728779+0000","last_became_peered":"2024-09-13T17:42:50.728779+0000","last_unstale":"2024-09-13T17:42:50.731410+0000","last_undegraded":"2024-09-13T17:42:50.731410+0000","last_fullsized":"2024-09-13T17:42:50.731410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691690+0000","last_change":"2024-09-13T17:42:50.691690+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691690+0000","last_undegraded":"2024-09-13T17:42:50.691690+0000","last_fullsized":"2024-09-13T17:42:50.691690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729125+0000","last_change":"2024-09-13T17:42:50.714804+0000","last_active":"2024-09-13T17:42:50.729125+0000","last_peered":"2024-09-13T17:42:50.729125+0000","last_clean":"2024-09-13T17:42:50.729125+0000","last_became_active":"2024-09-13T17:42:50.711927+0000","last_became_peered":"2024-09-13T17:42:50.711927+0000","last_unstale":"2024-09-13T17:42:50.729125+0000","last_undegraded":"2024-09-13T17:42:50.729125+0000","last_fullsized":"2024-09-13T17:42:50.729125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:04.749440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689421+0000","last_change":"2024-09-13T17:42:50.689421+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689421+0000","last_undegraded":"2024-09-13T17:42:50.689421+0000","last_fullsized":"2024-09-13T17:42:50.689421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729364+0000","last_change":"2024-09-13T17:42:50.716142+0000","last_active":"2024-09-13T17:42:50.729364+0000","last_peered":"2024-09-13T17:42:50.729364+0000","last_clean":"2024-09-13T17:42:50.729364+0000","last_became_active":"2024-09-13T17:42:50.715549+0000","last_became_peered":"2024-09-13T17:42:50.715549+0000","last_unstale":"2024-09-13T17:42:50.729364+0000","last_undegraded":"2024-09-13T17:42:50.729364+0000","last_fullsized":"2024-09-13T17:42:50.729364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691550+0000","last_change":"2024-09-13T17:42:50.691550+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691550+0000","last_undegraded":"2024-09-13T17:42:50.691550+0000","last_fullsized":"2024-09-13T17:42:50.691550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728587+0000","last_change":"2024-09-13T17:42:50.707353+0000","last_active":"2024-09-13T17:42:50.728587+0000","last_peered":"2024-09-13T17:42:50.728587+0000","last_clean":"2024-09-13T17:42:50.728587+0000","last_became_active":"2024-09-13T17:42:50.706452+0000","last_became_peered":"2024-09-13T17:42:50.706452+0000","last_unstale":"2024-09-13T17:42:50.728587+0000","last_undegraded":"2024-09-13T17:42:50.728587+0000","last_fullsized":"2024-09-13T17:42:50.728587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:15.294714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694956+0000","last_change":"2024-09-13T17:42:50.694956+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694956+0000","last_undegraded":"2024-09-13T17:42:50.694956+0000","last_fullsized":"2024-09-13T17:42:50.694956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731520+0000","last_change":"2024-09-13T17:42:50.730051+0000","last_active":"2024-09-13T17:42:50.731520+0000","last_peered":"2024-09-13T17:42:50.731520+0000","last_clean":"2024-09-13T17:42:50.731520+0000","last_became_active":"2024-09-13T17:42:50.729726+0000","last_became_peered":"2024-09-13T17:42:50.729726+0000","last_unstale":"2024-09-13T17:42:50.731520+0000","last_undegraded":"2024-09-13T17:42:50.731520+0000","last_fullsized":"2024-09-13T17:42:50.731520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:20.073192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691599+0000","last_change":"2024-09-13T17:42:50.691599+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691599+0000","last_undegraded":"2024-09-13T17:42:50.691599+0000","last_fullsized":"2024-09-13T17:42:50.691599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729188+0000","last_change":"2024-09-13T17:42:50.717327+0000","last_active":"2024-09-13T17:42:50.729188+0000","last_peered":"2024-09-13T17:42:50.729188+0000","last_clean":"2024-09-13T17:42:50.729188+0000","last_became_active":"2024-09-13T17:42:50.716762+0000","last_became_peered":"2024-09-13T17:42:50.716762+0000","last_unstale":"2024-09-13T17:42:50.729188+0000","last_undegraded":"2024-09-13T17:42:50.729188+0000","last_fullsized":"2024-09-13T17:42:50.729188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:17.303267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691810+0000","last_change":"2024-09-13T17:42:50.691810+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691810+0000","last_undegraded":"2024-09-13T17:42:50.691810+0000","last_fullsized":"2024-09-13T17:42:50.691810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717516+0000","last_change":"2024-09-13T17:42:50.711721+0000","last_active":"2024-09-13T17:42:50.717516+0000","last_peered":"2024-09-13T17:42:50.717516+0000","last_clean":"2024-09-13T17:42:50.717516+0000","last_became_active":"2024-09-13T17:42:50.711072+0000","last_became_peered":"2024-09-13T17:42:50.711072+0000","last_unstale":"2024-09-13T17:42:50.717516+0000","last_undegraded":"2024-09-13T17:42:50.717516+0000","last_fullsized":"2024-09-13T17:42:50.717516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:53.463804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693573+0000","last_change":"2024-09-13T17:42:50.693573+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693573+0000","last_undegraded":"2024-09-13T17:42:50.693573+0000","last_fullsized":"2024-09-13T17:42:50.693573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729218+0000","last_change":"2024-09-13T17:42:50.718435+0000","last_active":"2024-09-13T17:42:50.729218+0000","last_peered":"2024-09-13T17:42:50.729218+0000","last_clean":"2024-09-13T17:42:50.729218+0000","last_became_active":"2024-09-13T17:42:50.712419+0000","last_became_peered":"2024-09-13T17:42:50.712419+0000","last_unstale":"2024-09-13T17:42:50.729218+0000","last_undegraded":"2024-09-13T17:42:50.729218+0000","last_fullsized":"2024-09-13T17:42:50.729218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:34.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691245+0000","last_change":"2024-09-13T17:42:50.691245+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691245+0000","last_undegraded":"2024-09-13T17:42:50.691245+0000","last_fullsized":"2024-09-13T17:42:50.691245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.726179+0000","last_change":"2024-09-13T17:42:50.724850+0000","last_active":"2024-09-13T17:42:50.726179+0000","last_peered":"2024-09-13T17:42:50.726179+0000","last_clean":"2024-09-13T17:42:50.726179+0000","last_became_active":"2024-09-13T17:42:50.724216+0000","last_became_peered":"2024-09-13T17:42:50.724216+0000","last_unstale":"2024-09-13T17:42:50.726179+0000","last_undegraded":"2024-09-13T17:42:50.726179+0000","last_fullsized":"2024-09-13T17:42:50.726179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689035+0000","last_change":"2024-09-13T17:42:50.689035+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689035+0000","last_undegraded":"2024-09-13T17:42:50.689035+0000","last_fullsized":"2024-09-13T17:42:50.689035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.722805+0000","last_change":"2024-09-13T17:42:50.717580+0000","last_active":"2024-09-13T17:42:50.722805+0000","last_peered":"2024-09-13T17:42:50.722805+0000","last_clean":"2024-09-13T17:42:50.722805+0000","last_became_active":"2024-09-13T17:42:50.716737+0000","last_became_peered":"2024-09-13T17:42:50.716737+0000","last_unstale":"2024-09-13T17:42:50.722805+0000","last_undegraded":"2024-09-13T17:42:50.722805+0000","last_fullsized":"2024-09-13T17:42:50.722805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:44.217628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691862+0000","last_change":"2024-09-13T17:42:50.691862+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691862+0000","last_undegraded":"2024-09-13T17:42:50.691862+0000","last_fullsized":"2024-09-13T17:42:50.691862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729058+0000","last_change":"2024-09-13T17:42:50.712393+0000","last_active":"2024-09-13T17:42:50.729058+0000","last_peered":"2024-09-13T17:42:50.729058+0000","last_clean":"2024-09-13T17:42:50.729058+0000","last_became_active":"2024-09-13T17:42:50.712046+0000","last_became_peered":"2024-09-13T17:42:50.712046+0000","last_unstale":"2024-09-13T17:42:50.729058+0000","last_undegraded":"2024-09-13T17:42:50.729058+0000","last_fullsized":"2024-09-13T17:42:50.729058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:49.680242+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:49.680242+0000","last_peered":"2024-09-13T17:42:49.680242+0000","last_clean":"2024-09-13T17:42:49.680242+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:49.680242+0000","last_undegraded":"2024-09-13T17:42:49.680242+0000","last_fullsized":"2024-09-13T17:42:49.680242+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738383,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":63322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":653690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:55.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-09-13T17:42:55.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.697844+0000', 'last_change': '2024-09-13T17:42:50.697844+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.697844+0000', 'last_undegraded': '2024-09-13T17:42:50.697844+0000', 'last_fullsized': '2024-09-13T17:42:50.697844+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.700402+0000', 'last_change': '2024-09-13T17:42:50.700402+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.700402+0000', 'last_undegraded': '2024-09-13T17:42:50.700402+0000', 'last_fullsized': '2024-09-13T17:42:50.700402+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.708801+0000', 'last_change': '2024-09-13T17:42:50.708801+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.708801+0000', 'last_undegraded': '2024-09-13T17:42:50.708801+0000', 'last_fullsized': '2024-09-13T17:42:50.708801+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.710847+0000', 'last_change': '2024-09-13T17:42:50.710847+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.710847+0000', 'last_undegraded': '2024-09-13T17:42:50.710847+0000', 'last_fullsized': '2024-09-13T17:42:50.710847+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.698091+0000', 'last_change': '2024-09-13T17:42:50.698091+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.698091+0000', 'last_undegraded': '2024-09-13T17:42:50.698091+0000', 'last_fullsized': '2024-09-13T17:42:50.698091+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-09-13T17:42:55.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.708343+0000', 'last_change': '2024-09-13T17:42:50.708343+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.708343+0000', 'last_undegraded': '2024-09-13T17:42:50.708343+0000', 'last_fullsized': '2024-09-13T17:42:50.708343+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.698504+0000', 'last_change': '2024-09-13T17:42:50.698504+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.698504+0000', 'last_undegraded': '2024-09-13T17:42:50.698504+0000', 'last_fullsized': '2024-09-13T17:42:50.698504+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.697639+0000', 'last_change': '2024-09-13T17:42:50.697639+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.697639+0000', 'last_undegraded': '2024-09-13T17:42:50.697639+0000', 'last_fullsized': '2024-09-13T17:42:50.697639+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.708360+0000', 'last_change': '2024-09-13T17:42:50.708360+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.708360+0000', 'last_undegraded': '2024-09-13T17:42:50.708360+0000', 'last_fullsized': '2024-09-13T17:42:50.708360+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.707967+0000', 'last_change': '2024-09-13T17:42:50.707967+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.707967+0000', 'last_undegraded': '2024-09-13T17:42:50.707967+0000', 'last_fullsized': '2024-09-13T17:42:50.707967+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-09-13T17:42:55.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.705349+0000', 'last_change': '2024-09-13T17:42:50.705349+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.705349+0000', 'last_undegraded': '2024-09-13T17:42:50.705349+0000', 'last_fullsized': '2024-09-13T17:42:50.705349+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.708384+0000', 'last_change': '2024-09-13T17:42:50.708384+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.708384+0000', 'last_undegraded': '2024-09-13T17:42:50.708384+0000', 'last_fullsized': '2024-09-13T17:42:50.708384+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696230+0000', 'last_change': '2024-09-13T17:42:50.696230+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696230+0000', 'last_undegraded': '2024-09-13T17:42:50.696230+0000', 'last_fullsized': '2024-09-13T17:42:50.696230+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.705522+0000', 'last_change': '2024-09-13T17:42:50.705522+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.705522+0000', 'last_undegraded': '2024-09-13T17:42:50.705522+0000', 'last_fullsized': '2024-09-13T17:42:50.705522+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-09-13T17:42:55.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.699632+0000', 'last_change': '2024-09-13T17:42:50.699632+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.699632+0000', 'last_undegraded': '2024-09-13T17:42:50.699632+0000', 'last_fullsized': '2024-09-13T17:42:50.699632+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.699190+0000', 'last_change': '2024-09-13T17:42:50.699190+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.699190+0000', 'last_undegraded': '2024-09-13T17:42:50.699190+0000', 'last_fullsized': '2024-09-13T17:42:50.699190+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.699984+0000', 'last_change': '2024-09-13T17:42:50.699984+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.699984+0000', 'last_undegraded': '2024-09-13T17:42:50.699984+0000', 'last_fullsized': '2024-09-13T17:42:50.699984+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.699637+0000', 'last_change': '2024-09-13T17:42:50.699637+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.699637+0000', 'last_undegraded': '2024-09-13T17:42:50.699637+0000', 'last_fullsized': '2024-09-13T17:42:50.699637+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.695823+0000', 'last_change': '2024-09-13T17:42:50.695823+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.695823+0000', 'last_undegraded': '2024-09-13T17:42:50.695823+0000', 'last_fullsized': '2024-09-13T17:42:50.695823+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-09-13T17:42:55.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.697278+0000', 'last_change': '2024-09-13T17:42:50.697278+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.697278+0000', 'last_undegraded': '2024-09-13T17:42:50.697278+0000', 'last_fullsized': '2024-09-13T17:42:50.697278+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.700418+0000', 'last_change': '2024-09-13T17:42:50.700418+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.700418+0000', 'last_undegraded': '2024-09-13T17:42:50.700418+0000', 'last_fullsized': '2024-09-13T17:42:50.700418+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696790+0000', 'last_change': '2024-09-13T17:42:50.696790+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696790+0000', 'last_undegraded': '2024-09-13T17:42:50.696790+0000', 'last_fullsized': '2024-09-13T17:42:50.696790+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.695279+0000', 'last_change': '2024-09-13T17:42:50.695279+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.695279+0000', 'last_undegraded': '2024-09-13T17:42:50.695279+0000', 'last_fullsized': '2024-09-13T17:42:50.695279+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.697222+0000', 'last_change': '2024-09-13T17:42:50.697222+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.697222+0000', 'last_undegraded': '2024-09-13T17:42:50.697222+0000', 'last_fullsized': '2024-09-13T17:42:50.697222+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.705812+0000', 'last_change': '2024-09-13T17:42:50.705812+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.705812+0000', 'last_undegraded': '2024-09-13T17:42:50.705812+0000', 'last_fullsized': '2024-09-13T17:42:50.705812+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-09-13T17:42:55.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.704785+0000', 'last_change': '2024-09-13T17:42:50.704785+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.704785+0000', 'last_undegraded': '2024-09-13T17:42:50.704785+0000', 'last_fullsized': '2024-09-13T17:42:50.704785+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.701412+0000', 'last_change': '2024-09-13T17:42:50.701412+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.701412+0000', 'last_undegraded': '2024-09-13T17:42:50.701412+0000', 'last_fullsized': '2024-09-13T17:42:50.701412+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696233+0000', 'last_change': '2024-09-13T17:42:50.696233+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696233+0000', 'last_undegraded': '2024-09-13T17:42:50.696233+0000', 'last_fullsized': '2024-09-13T17:42:50.696233+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.695759+0000', 'last_change': '2024-09-13T17:42:50.695759+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.695759+0000', 'last_undegraded': '2024-09-13T17:42:50.695759+0000', 'last_fullsized': '2024-09-13T17:42:50.695759+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694091+0000', 'last_change': '2024-09-13T17:42:50.694091+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694091+0000', 'last_undegraded': '2024-09-13T17:42:50.694091+0000', 'last_fullsized': '2024-09-13T17:42:50.694091+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696667+0000', 'last_change': '2024-09-13T17:42:50.696667+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696667+0000', 'last_undegraded': '2024-09-13T17:42:50.696667+0000', 'last_fullsized': '2024-09-13T17:42:50.696667+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696514+0000', 'last_change': '2024-09-13T17:42:50.696514+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696514+0000', 'last_undegraded': '2024-09-13T17:42:50.696514+0000', 'last_fullsized': '2024-09-13T17:42:50.696514+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694989+0000', 'last_change': '2024-09-13T17:42:50.694989+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694989+0000', 'last_undegraded': '2024-09-13T17:42:50.694989+0000', 'last_fullsized': '2024-09-13T17:42:50.694989+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696149+0000', 'last_change': '2024-09-13T17:42:50.696149+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696149+0000', 'last_undegraded': '2024-09-13T17:42:50.696149+0000', 'last_fullsized': '2024-09-13T17:42:50.696149+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694460+0000', 'last_change': '2024-09-13T17:42:50.694460+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694460+0000', 'last_undegraded': '2024-09-13T17:42:50.694460+0000', 'last_fullsized': '2024-09-13T17:42:50.694460+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.700402+0000', 'last_change': '2024-09-13T17:42:50.700402+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.700402+0000', 'last_undegraded': '2024-09-13T17:42:50.700402+0000', 'last_fullsized': '2024-09-13T17:42:50.700402+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-09-13T17:42:55.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693222+0000', 'last_change': '2024-09-13T17:42:50.693222+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693222+0000', 'last_undegraded': '2024-09-13T17:42:50.693222+0000', 'last_fullsized': '2024-09-13T17:42:50.693222+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.695070+0000', 'last_change': '2024-09-13T17:42:50.695070+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.695070+0000', 'last_undegraded': '2024-09-13T17:42:50.695070+0000', 'last_fullsized': '2024-09-13T17:42:50.695070+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693187+0000', 'last_change': '2024-09-13T17:42:50.693187+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693187+0000', 'last_undegraded': '2024-09-13T17:42:50.693187+0000', 'last_fullsized': '2024-09-13T17:42:50.693187+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693746+0000', 'last_change': '2024-09-13T17:42:50.693746+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693746+0000', 'last_undegraded': '2024-09-13T17:42:50.693746+0000', 'last_fullsized': '2024-09-13T17:42:50.693746+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.690548+0000', 'last_change': '2024-09-13T17:42:50.690548+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.690548+0000', 'last_undegraded': '2024-09-13T17:42:50.690548+0000', 'last_fullsized': '2024-09-13T17:42:50.690548+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694137+0000', 'last_change': '2024-09-13T17:42:50.694137+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694137+0000', 'last_undegraded': '2024-09-13T17:42:50.694137+0000', 'last_fullsized': '2024-09-13T17:42:50.694137+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693969+0000', 'last_change': '2024-09-13T17:42:50.693969+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693969+0000', 'last_undegraded': '2024-09-13T17:42:50.693969+0000', 'last_fullsized': '2024-09-13T17:42:50.693969+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694860+0000', 'last_change': '2024-09-13T17:42:50.694860+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694860+0000', 'last_undegraded': '2024-09-13T17:42:50.694860+0000', 'last_fullsized': '2024-09-13T17:42:50.694860+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.692348+0000', 'last_change': '2024-09-13T17:42:50.692348+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.692348+0000', 'last_undegraded': '2024-09-13T17:42:50.692348+0000', 'last_fullsized': '2024-09-13T17:42:50.692348+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693342+0000', 'last_change': '2024-09-13T17:42:50.693342+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693342+0000', 'last_undegraded': '2024-09-13T17:42:50.693342+0000', 'last_fullsized': '2024-09-13T17:42:50.693342+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.692083+0000', 'last_change': '2024-09-13T17:42:50.692083+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.692083+0000', 'last_undegraded': '2024-09-13T17:42:50.692083+0000', 'last_fullsized': '2024-09-13T17:42:50.692083+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-09-13T17:42:55.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693304+0000', 'last_change': '2024-09-13T17:42:50.693304+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693304+0000', 'last_undegraded': '2024-09-13T17:42:50.693304+0000', 'last_fullsized': '2024-09-13T17:42:50.693304+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696713+0000', 'last_change': '2024-09-13T17:42:50.696713+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696713+0000', 'last_undegraded': '2024-09-13T17:42:50.696713+0000', 'last_fullsized': '2024-09-13T17:42:50.696713+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693033+0000', 'last_change': '2024-09-13T17:42:50.693033+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693033+0000', 'last_undegraded': '2024-09-13T17:42:50.693033+0000', 'last_fullsized': '2024-09-13T17:42:50.693033+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.687292+0000', 'last_change': '2024-09-13T17:42:50.687292+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.687292+0000', 'last_undegraded': '2024-09-13T17:42:50.687292+0000', 'last_fullsized': '2024-09-13T17:42:50.687292+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.696247+0000', 'last_change': '2024-09-13T17:42:50.696247+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.696247+0000', 'last_undegraded': '2024-09-13T17:42:50.696247+0000', 'last_fullsized': '2024-09-13T17:42:50.696247+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-09-13T17:42:55.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691952+0000', 'last_change': '2024-09-13T17:42:50.691952+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691952+0000', 'last_undegraded': '2024-09-13T17:42:50.691952+0000', 'last_fullsized': '2024-09-13T17:42:50.691952+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.687270+0000', 'last_change': '2024-09-13T17:42:50.687270+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.687270+0000', 'last_undegraded': '2024-09-13T17:42:50.687270+0000', 'last_fullsized': '2024-09-13T17:42:50.687270+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691690+0000', 'last_change': '2024-09-13T17:42:50.691690+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691690+0000', 'last_undegraded': '2024-09-13T17:42:50.691690+0000', 'last_fullsized': '2024-09-13T17:42:50.691690+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.689421+0000', 'last_change': '2024-09-13T17:42:50.689421+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.689421+0000', 'last_undegraded': '2024-09-13T17:42:50.689421+0000', 'last_fullsized': '2024-09-13T17:42:50.689421+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691550+0000', 'last_change': '2024-09-13T17:42:50.691550+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691550+0000', 'last_undegraded': '2024-09-13T17:42:50.691550+0000', 'last_fullsized': '2024-09-13T17:42:50.691550+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-09-13T17:42:55.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.694956+0000', 'last_change': '2024-09-13T17:42:50.694956+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.694956+0000', 'last_undegraded': '2024-09-13T17:42:50.694956+0000', 'last_fullsized': '2024-09-13T17:42:50.694956+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691599+0000', 'last_change': '2024-09-13T17:42:50.691599+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691599+0000', 'last_undegraded': '2024-09-13T17:42:50.691599+0000', 'last_fullsized': '2024-09-13T17:42:50.691599+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691810+0000', 'last_change': '2024-09-13T17:42:50.691810+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691810+0000', 'last_undegraded': '2024-09-13T17:42:50.691810+0000', 'last_fullsized': '2024-09-13T17:42:50.691810+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.693573+0000', 'last_change': '2024-09-13T17:42:50.693573+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.693573+0000', 'last_undegraded': '2024-09-13T17:42:50.693573+0000', 'last_fullsized': '2024-09-13T17:42:50.693573+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691245+0000', 'last_change': '2024-09-13T17:42:50.691245+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691245+0000', 'last_undegraded': '2024-09-13T17:42:50.691245+0000', 'last_fullsized': '2024-09-13T17:42:50.691245+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.689035+0000', 'last_change': '2024-09-13T17:42:50.689035+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.689035+0000', 'last_undegraded': '2024-09-13T17:42:50.689035+0000', 'last_fullsized': '2024-09-13T17:42:50.689035+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-13T17:42:55.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-09-13T17:42:55.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 23, 'state': 'creating+peering', 'last_fresh': '2024-09-13T17:42:50.691862+0000', 'last_change': '2024-09-13T17:42:50.691862+0000', 'last_active': '2024-09-13T17:42:50.665253+0000', 'last_peered': '2024-09-13T17:42:50.665253+0000', 'last_clean': '2024-09-13T17:42:50.665253+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-13T17:42:50.691862+0000', 'last_undegraded': '2024-09-13T17:42:50.691862+0000', 'last_fullsized': '2024-09-13T17:42:50.691862+0000', 'mapping_epoch': 23, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 23, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'last_clean_scrub_stamp': '2024-09-13T17:42:50.665253+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-13T17:42:55.728 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T17:42:55.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efee2698640 1 Processor -- start 2024-09-13T17:42:55.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efee2698640 1 -- start start 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efee2698640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efedc06f6d0 0x7efedc134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efee2698640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efedc06e130 con 0x7efedc09dc20 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efee2698640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efedc0a6750 con 0x7efedc06f6d0 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efedc06f6d0 0x7efedc134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efedc06f6d0 0x7efedc134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38916/0 (socket says 172.21.15.96:38916) 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedb7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efedc09dc20 0x7efedc134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44362/0 (socket says 172.21.15.96:44362) 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedbfff640 1 -- 172.21.15.96:0/58029704 learned_addr learned my addr 172.21.15.96:0/58029704 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedbfff640 1 -- 172.21.15.96:0/58029704 >> v1:172.21.15.96:6789/0 conn(0x7efedc09dc20 legacy=0x7efedc134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.867+0000 7efedbfff640 1 -- 172.21.15.96:0/58029704 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc0712e0 con 0x7efedc06f6d0 2024-09-13T17:42:55.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efedbfff640 1 --2- 172.21.15.96:0/58029704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc134ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efecc0097f0 tx=0x7efecc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3a0d4cc997a268 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efedaffd640 1 -- 172.21.15.96:0/58029704 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efecc0ab400 con 0x7efedc06f6d0 2024-09-13T17:42:55.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efedaffd640 1 -- 172.21.15.96:0/58029704 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc0a7050 con 0x7efedc06f6d0 2024-09-13T17:42:55.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efedaffd640 1 -- 172.21.15.96:0/58029704 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efecc0ab6b0 con 0x7efedc06f6d0 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- 172.21.15.96:0/58029704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 msgr2=0x7efedc134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 --2- 172.21.15.96:0/58029704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc134ab0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7efecc0097f0 tx=0x7efecc01c450 comp rx=0 tx=0).stop 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- 172.21.15.96:0/58029704 shutdown_connections 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 --2- 172.21.15.96:0/58029704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc134ab0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- 172.21.15.96:0/58029704 >> 172.21.15.96:0/58029704 conn(0x7efedc098a60 msgr2=0x7efedc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- 172.21.15.96:0/58029704 shutdown_connections 2024-09-13T17:42:55.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- 172.21.15.96:0/58029704 wait complete. 2024-09-13T17:42:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 Processor -- start 2024-09-13T17:42:55.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.871+0000 7efee2698640 1 -- start start 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efee2698640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efee2698640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc0a6160 con 0x7efedc06f6d0 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38920/0 (socket says 172.21.15.96:38920) 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedbfff640 1 -- 172.21.15.96:0/1308521697 learned_addr learned my addr 172.21.15.96:0/1308521697 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedbfff640 1 -- 172.21.15.96:0/1308521697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0094d0 con 0x7efedc06f6d0 2024-09-13T17:42:55.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedbfff640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efecc01a080 tx=0x7efecc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efecc0b6420 con 0x7efedc06f6d0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc0b6580 con 0x7efedc06f6d0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc0a1e40 con 0x7efedc06f6d0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efecc0b6840 con 0x7efedc06f6d0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc0a30c0 con 0x7efedc06f6d0 2024-09-13T17:42:55.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc134ab0 con 0x7efedc06f6d0 2024-09-13T17:42:55.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7efecc0b2040 con 0x7efedc06f6d0 2024-09-13T17:42:55.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 0x7efec8037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:55.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7efecc0e1fb0 con 0x7efedc06f6d0 2024-09-13T17:42:55.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedb7fe640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 0x7efec8037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:55.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.875+0000 7efedb7fe640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 0x7efec8037910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efed0001600 tx=0x7efed0007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:55.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:55.879+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efecc0b69a0 con 0x7efedc06f6d0 2024-09-13T17:42:56.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.091+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efedc06a1b0 con 0x7efedc06f6d0 2024-09-13T17:42:56.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efed8ff9640 1 -- 172.21.15.96:0/1308521697 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1309 (secure 0 0 0) 0x7efecc0b0020 con 0x7efedc06f6d0 2024-09-13T17:42:56.098 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:56.098 INFO:teuthology.orchestra.run.smithi096.stdout:{"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":75,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":25,"num_osds":3,"num_up_osds":3,"osd_up_since":1726249310,"num_in_osds":3,"osd_in_since":1726249301,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":84795392,"bytes_avail":289825497088,"bytes_total":289910292480,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":14,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4488}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-13T17:42:51.808026+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 msgr2=0x7efec8037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 0x7efec8037910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7efed0001600 tx=0x7efed0007a50 comp rx=0 tx=0).stop 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 msgr2=0x7efedc0a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7efecc01a080 tx=0x7efecc01ac90 comp rx=0 tx=0).stop 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 shutdown_connections 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7efec8035450 0x7efec8037910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 --2- 172.21.15.96:0/1308521697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efedc06f6d0 0x7efedc0a4270 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 >> 172.21.15.96:0/1308521697 conn(0x7efedc098a60 msgr2=0x7efedc133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 shutdown_connections 2024-09-13T17:42:56.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.095+0000 7efee2698640 1 -- 172.21.15.96:0/1308521697 wait complete. 2024-09-13T17:42:56.117 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-09-13T17:42:56.117 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:56.196 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:42:56.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f360e21a640 1 Processor -- start 2024-09-13T17:42:56.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f360e21a640 1 -- start start 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f360e21a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f360809d6f0 0x7f3608138340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f360e21a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3608138110 con 0x7f3608138880 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f360e21a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f360809cf90 con 0x7f360809d6f0 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f3606ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3608138880 0x7f3608134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44366/0 (socket says 172.21.15.96:44366) 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f3606ffd640 1 -- 172.21.15.96:0/308854405 learned_addr learned my addr 172.21.15.96:0/308854405 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:56.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f36077fe640 1 --2- 172.21.15.96:0/308854405 >> v2:172.21.15.96:3300/0 conn(0x7f360809d6f0 0x7f3608138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:56.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.231+0000 7f36067fc640 1 -- 172.21.15.96:0/308854405 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2185746564 0 0) 0x7f3608138110 con 0x7f3608138880 2024-09-13T17:42:56.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36067fc640 1 -- 172.21.15.96:0/308854405 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35f0003530 con 0x7f3608138880 2024-09-13T17:42:56.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36077fe640 1 -- 172.21.15.96:0/308854405 >> v1:172.21.15.96:6789/0 conn(0x7f3608138880 legacy=0x7f3608134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36077fe640 1 -- 172.21.15.96:0/308854405 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360809dad0 con 0x7f360809d6f0 2024-09-13T17:42:56.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36077fe640 1 --2- 172.21.15.96:0/308854405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f3608138340 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009740 tx=0x7f35f801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854ce42de0ff91ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:56.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36067fc640 1 -- 172.21.15.96:0/308854405 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3608135570 con 0x7f360809d6f0 2024-09-13T17:42:56.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36067fc640 1 -- 172.21.15.96:0/308854405 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f80a7050 con 0x7f360809d6f0 2024-09-13T17:42:56.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f36067fc640 1 -- 172.21.15.96:0/308854405 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f80ab660 con 0x7f360809d6f0 2024-09-13T17:42:56.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 -- 172.21.15.96:0/308854405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 msgr2=0x7f3608138340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 --2- 172.21.15.96:0/308854405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f3608138340 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009740 tx=0x7f35f801c390 comp rx=0 tx=0).stop 2024-09-13T17:42:56.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 -- 172.21.15.96:0/308854405 shutdown_connections 2024-09-13T17:42:56.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 --2- 172.21.15.96:0/308854405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f3608138340 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 -- 172.21.15.96:0/308854405 >> 172.21.15.96:0/308854405 conn(0x7f360806bbd0 msgr2=0x7f360806bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:56.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 -- 172.21.15.96:0/308854405 shutdown_connections 2024-09-13T17:42:56.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.235+0000 7f360e21a640 1 -- 172.21.15.96:0/308854405 wait complete. 2024-09-13T17:42:56.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 Processor -- start 2024-09-13T17:42:56.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 -- start start 2024-09-13T17:42:56.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36080a2930 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f36077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f36077fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38938/0 (socket says 172.21.15.96:38938) 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f36077fe640 1 -- 172.21.15.96:0/3008479395 learned_addr learned my addr 172.21.15.96:0/3008479395 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f36077fe640 1 -- 172.21.15.96:0/3008479395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8009420 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f36077fe640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f35f80ae040 tx=0x7f35f8019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f80abdf0 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f80b4410 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f35f80b46b0 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3608137960 con 0x7f360809d6f0 2024-09-13T17:42:56.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36080a3d60 con 0x7f360809d6f0 2024-09-13T17:42:56.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f35f80b2040 con 0x7f360809d6f0 2024-09-13T17:42:56.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3608138340 con 0x7f360809d6f0 2024-09-13T17:42:56.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 0x7f35f4037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:56.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f35f80e15b0 con 0x7f360809d6f0 2024-09-13T17:42:56.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f3606ffd640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 0x7f35f4037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:56.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.239+0000 7f3606ffd640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 0x7f35f4037910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000f70 tx=0x7f35fc0010b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:56.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.247+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f35f80c5d20 con 0x7f360809d6f0 2024-09-13T17:42:56.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.403+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36080a3ef0 con 0x7f35f4035450 2024-09-13T17:42:56.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.411+0000 7f35effff640 1 -- 172.21.15.96:0/3008479395 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+244448 (secure 0 0 0) 0x7f35fc0058e0 con 0x7f35f4035450 2024-09-13T17:42:56.418 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:56.419 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 msgr2=0x7f35f4037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 0x7f35f4037910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f35fc000f70 tx=0x7f35fc0010b0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 msgr2=0x7f36081cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f35f80ae040 tx=0x7f35f8019130 comp rx=0 tx=0).stop 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 shutdown_connections 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f35f4035450 0x7f35f4037910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 --2- 172.21.15.96:0/3008479395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f360809d6f0 0x7f36081cc640 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 >> 172.21.15.96:0/3008479395 conn(0x7f360806bbd0 msgr2=0x7f3608131190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 shutdown_connections 2024-09-13T17:42:56.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:56.415+0000 7f360e21a640 1 -- 172.21.15.96:0/3008479395 wait complete. 2024-09-13T17:42:56.436 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-13T17:42:55.809006+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82808,"kb_used_data":1744,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032712,"statfs":{"total":289910292480,"available":289825497088,"internally_reserved":0,"allocated":1785856,"data_stored":1374667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001349"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697844+0000","last_change":"2024-09-13T17:42:50.697844+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697844+0000","last_undegraded":"2024-09-13T17:42:50.697844+0000","last_fullsized":"2024-09-13T17:42:50.697844+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708801+0000","last_change":"2024-09-13T17:42:50.708801+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708801+0000","last_undegraded":"2024-09-13T17:42:50.708801+0000","last_fullsized":"2024-09-13T17:42:50.708801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.710847+0000","last_change":"2024-09-13T17:42:50.710847+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.710847+0000","last_undegraded":"2024-09-13T17:42:50.710847+0000","last_fullsized":"2024-09-13T17:42:50.710847+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698091+0000","last_change":"2024-09-13T17:42:50.698091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698091+0000","last_undegraded":"2024-09-13T17:42:50.698091+0000","last_fullsized":"2024-09-13T17:42:50.698091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708343+0000","last_change":"2024-09-13T17:42:50.708343+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708343+0000","last_undegraded":"2024-09-13T17:42:50.708343+0000","last_fullsized":"2024-09-13T17:42:50.708343+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.698504+0000","last_change":"2024-09-13T17:42:50.698504+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.698504+0000","last_undegraded":"2024-09-13T17:42:50.698504+0000","last_fullsized":"2024-09-13T17:42:50.698504+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697639+0000","last_change":"2024-09-13T17:42:50.697639+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697639+0000","last_undegraded":"2024-09-13T17:42:50.697639+0000","last_fullsized":"2024-09-13T17:42:50.697639+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708360+0000","last_change":"2024-09-13T17:42:50.708360+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708360+0000","last_undegraded":"2024-09-13T17:42:50.708360+0000","last_fullsized":"2024-09-13T17:42:50.708360+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.707967+0000","last_change":"2024-09-13T17:42:50.707967+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.707967+0000","last_undegraded":"2024-09-13T17:42:50.707967+0000","last_fullsized":"2024-09-13T17:42:50.707967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705349+0000","last_change":"2024-09-13T17:42:50.705349+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705349+0000","last_undegraded":"2024-09-13T17:42:50.705349+0000","last_fullsized":"2024-09-13T17:42:50.705349+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.708384+0000","last_change":"2024-09-13T17:42:50.708384+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.708384+0000","last_undegraded":"2024-09-13T17:42:50.708384+0000","last_fullsized":"2024-09-13T17:42:50.708384+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696230+0000","last_change":"2024-09-13T17:42:50.696230+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696230+0000","last_undegraded":"2024-09-13T17:42:50.696230+0000","last_fullsized":"2024-09-13T17:42:50.696230+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705522+0000","last_change":"2024-09-13T17:42:50.705522+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705522+0000","last_undegraded":"2024-09-13T17:42:50.705522+0000","last_fullsized":"2024-09-13T17:42:50.705522+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699632+0000","last_change":"2024-09-13T17:42:50.699632+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699632+0000","last_undegraded":"2024-09-13T17:42:50.699632+0000","last_fullsized":"2024-09-13T17:42:50.699632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699190+0000","last_change":"2024-09-13T17:42:50.699190+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699190+0000","last_undegraded":"2024-09-13T17:42:50.699190+0000","last_fullsized":"2024-09-13T17:42:50.699190+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699984+0000","last_change":"2024-09-13T17:42:50.699984+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699984+0000","last_undegraded":"2024-09-13T17:42:50.699984+0000","last_fullsized":"2024-09-13T17:42:50.699984+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.699637+0000","last_change":"2024-09-13T17:42:50.699637+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.699637+0000","last_undegraded":"2024-09-13T17:42:50.699637+0000","last_fullsized":"2024-09-13T17:42:50.699637+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695823+0000","last_change":"2024-09-13T17:42:50.695823+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695823+0000","last_undegraded":"2024-09-13T17:42:50.695823+0000","last_fullsized":"2024-09-13T17:42:50.695823+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697278+0000","last_change":"2024-09-13T17:42:50.697278+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697278+0000","last_undegraded":"2024-09-13T17:42:50.697278+0000","last_fullsized":"2024-09-13T17:42:50.697278+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700418+0000","last_change":"2024-09-13T17:42:50.700418+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700418+0000","last_undegraded":"2024-09-13T17:42:50.700418+0000","last_fullsized":"2024-09-13T17:42:50.700418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696790+0000","last_change":"2024-09-13T17:42:50.696790+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696790+0000","last_undegraded":"2024-09-13T17:42:50.696790+0000","last_fullsized":"2024-09-13T17:42:50.696790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695279+0000","last_change":"2024-09-13T17:42:50.695279+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695279+0000","last_undegraded":"2024-09-13T17:42:50.695279+0000","last_fullsized":"2024-09-13T17:42:50.695279+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.697222+0000","last_change":"2024-09-13T17:42:50.697222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.697222+0000","last_undegraded":"2024-09-13T17:42:50.697222+0000","last_fullsized":"2024-09-13T17:42:50.697222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.705812+0000","last_change":"2024-09-13T17:42:50.705812+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.705812+0000","last_undegraded":"2024-09-13T17:42:50.705812+0000","last_fullsized":"2024-09-13T17:42:50.705812+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.704785+0000","last_change":"2024-09-13T17:42:50.704785+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.704785+0000","last_undegraded":"2024-09-13T17:42:50.704785+0000","last_fullsized":"2024-09-13T17:42:50.704785+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.701412+0000","last_change":"2024-09-13T17:42:50.701412+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.701412+0000","last_undegraded":"2024-09-13T17:42:50.701412+0000","last_fullsized":"2024-09-13T17:42:50.701412+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696233+0000","last_change":"2024-09-13T17:42:50.696233+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696233+0000","last_undegraded":"2024-09-13T17:42:50.696233+0000","last_fullsized":"2024-09-13T17:42:50.696233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695759+0000","last_change":"2024-09-13T17:42:50.695759+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695759+0000","last_undegraded":"2024-09-13T17:42:50.695759+0000","last_fullsized":"2024-09-13T17:42:50.695759+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694091+0000","last_change":"2024-09-13T17:42:50.694091+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694091+0000","last_undegraded":"2024-09-13T17:42:50.694091+0000","last_fullsized":"2024-09-13T17:42:50.694091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696667+0000","last_change":"2024-09-13T17:42:50.696667+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696667+0000","last_undegraded":"2024-09-13T17:42:50.696667+0000","last_fullsized":"2024-09-13T17:42:50.696667+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696514+0000","last_change":"2024-09-13T17:42:50.696514+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696514+0000","last_undegraded":"2024-09-13T17:42:50.696514+0000","last_fullsized":"2024-09-13T17:42:50.696514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694989+0000","last_change":"2024-09-13T17:42:50.694989+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694989+0000","last_undegraded":"2024-09-13T17:42:50.694989+0000","last_fullsized":"2024-09-13T17:42:50.694989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731524+0000","last_change":"2024-09-13T17:42:50.729720+0000","last_active":"2024-09-13T17:42:50.731524+0000","last_peered":"2024-09-13T17:42:50.731524+0000","last_clean":"2024-09-13T17:42:50.731524+0000","last_became_active":"2024-09-13T17:42:50.729403+0000","last_became_peered":"2024-09-13T17:42:50.729403+0000","last_unstale":"2024-09-13T17:42:50.731524+0000","last_undegraded":"2024-09-13T17:42:50.731524+0000","last_fullsized":"2024-09-13T17:42:50.731524+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:02.933863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696149+0000","last_change":"2024-09-13T17:42:50.696149+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696149+0000","last_undegraded":"2024-09-13T17:42:50.696149+0000","last_fullsized":"2024-09-13T17:42:50.696149+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731455+0000","last_change":"2024-09-13T17:42:50.729650+0000","last_active":"2024-09-13T17:42:50.731455+0000","last_peered":"2024-09-13T17:42:50.731455+0000","last_clean":"2024-09-13T17:42:50.731455+0000","last_became_active":"2024-09-13T17:42:50.729126+0000","last_became_peered":"2024-09-13T17:42:50.729126+0000","last_unstale":"2024-09-13T17:42:50.731455+0000","last_undegraded":"2024-09-13T17:42:50.731455+0000","last_fullsized":"2024-09-13T17:42:50.731455+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:09.600109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694460+0000","last_change":"2024-09-13T17:42:50.694460+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694460+0000","last_undegraded":"2024-09-13T17:42:50.694460+0000","last_fullsized":"2024-09-13T17:42:50.694460+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728468+0000","last_change":"2024-09-13T17:42:50.727195+0000","last_active":"2024-09-13T17:42:50.728468+0000","last_peered":"2024-09-13T17:42:50.728468+0000","last_clean":"2024-09-13T17:42:50.728468+0000","last_became_active":"2024-09-13T17:42:50.726846+0000","last_became_peered":"2024-09-13T17:42:50.726846+0000","last_unstale":"2024-09-13T17:42:50.728468+0000","last_undegraded":"2024-09-13T17:42:50.728468+0000","last_fullsized":"2024-09-13T17:42:50.728468+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:11.620151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.700402+0000","last_change":"2024-09-13T17:42:50.700402+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.700402+0000","last_undegraded":"2024-09-13T17:42:50.700402+0000","last_fullsized":"2024-09-13T17:42:50.700402+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732055+0000","last_change":"2024-09-13T17:42:50.730528+0000","last_active":"2024-09-13T17:42:50.732055+0000","last_peered":"2024-09-13T17:42:50.732055+0000","last_clean":"2024-09-13T17:42:50.732055+0000","last_became_active":"2024-09-13T17:42:50.730190+0000","last_became_peered":"2024-09-13T17:42:50.730190+0000","last_unstale":"2024-09-13T17:42:50.732055+0000","last_undegraded":"2024-09-13T17:42:50.732055+0000","last_fullsized":"2024-09-13T17:42:50.732055+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693222+0000","last_change":"2024-09-13T17:42:50.693222+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693222+0000","last_undegraded":"2024-09-13T17:42:50.693222+0000","last_fullsized":"2024-09-13T17:42:50.693222+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731453+0000","last_change":"2024-09-13T17:42:50.729206+0000","last_active":"2024-09-13T17:42:50.731453+0000","last_peered":"2024-09-13T17:42:50.731453+0000","last_clean":"2024-09-13T17:42:50.731453+0000","last_became_active":"2024-09-13T17:42:50.728865+0000","last_became_peered":"2024-09-13T17:42:50.728865+0000","last_unstale":"2024-09-13T17:42:50.731453+0000","last_undegraded":"2024-09-13T17:42:50.731453+0000","last_fullsized":"2024-09-13T17:42:50.731453+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:09.981862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.695070+0000","last_change":"2024-09-13T17:42:50.695070+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.695070+0000","last_undegraded":"2024-09-13T17:42:50.695070+0000","last_fullsized":"2024-09-13T17:42:50.695070+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.725764+0000","last_change":"2024-09-13T17:42:50.719752+0000","last_active":"2024-09-13T17:42:50.725764+0000","last_peered":"2024-09-13T17:42:50.725764+0000","last_clean":"2024-09-13T17:42:50.725764+0000","last_became_active":"2024-09-13T17:42:50.719113+0000","last_became_peered":"2024-09-13T17:42:50.719113+0000","last_unstale":"2024-09-13T17:42:50.725764+0000","last_undegraded":"2024-09-13T17:42:50.725764+0000","last_fullsized":"2024-09-13T17:42:50.725764+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:14.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693187+0000","last_change":"2024-09-13T17:42:50.693187+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693187+0000","last_undegraded":"2024-09-13T17:42:50.693187+0000","last_fullsized":"2024-09-13T17:42:50.693187+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717741+0000","last_change":"2024-09-13T17:42:50.711236+0000","last_active":"2024-09-13T17:42:50.717741+0000","last_peered":"2024-09-13T17:42:50.717741+0000","last_clean":"2024-09-13T17:42:50.717741+0000","last_became_active":"2024-09-13T17:42:50.710880+0000","last_became_peered":"2024-09-13T17:42:50.710880+0000","last_unstale":"2024-09-13T17:42:50.717741+0000","last_undegraded":"2024-09-13T17:42:50.717741+0000","last_fullsized":"2024-09-13T17:42:50.717741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.473381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693746+0000","last_change":"2024-09-13T17:42:50.693746+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693746+0000","last_undegraded":"2024-09-13T17:42:50.693746+0000","last_fullsized":"2024-09-13T17:42:50.693746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732006+0000","last_change":"2024-09-13T17:42:50.729497+0000","last_active":"2024-09-13T17:42:50.732006+0000","last_peered":"2024-09-13T17:42:50.732006+0000","last_clean":"2024-09-13T17:42:50.732006+0000","last_became_active":"2024-09-13T17:42:50.728723+0000","last_became_peered":"2024-09-13T17:42:50.728723+0000","last_unstale":"2024-09-13T17:42:50.732006+0000","last_undegraded":"2024-09-13T17:42:50.732006+0000","last_fullsized":"2024-09-13T17:42:50.732006+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:36.241145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.690548+0000","last_change":"2024-09-13T17:42:50.690548+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.690548+0000","last_undegraded":"2024-09-13T17:42:50.690548+0000","last_fullsized":"2024-09-13T17:42:50.690548+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731541+0000","last_change":"2024-09-13T17:42:50.729826+0000","last_active":"2024-09-13T17:42:50.731541+0000","last_peered":"2024-09-13T17:42:50.731541+0000","last_clean":"2024-09-13T17:42:50.731541+0000","last_became_active":"2024-09-13T17:42:50.729515+0000","last_became_peered":"2024-09-13T17:42:50.729515+0000","last_unstale":"2024-09-13T17:42:50.731541+0000","last_undegraded":"2024-09-13T17:42:50.731541+0000","last_fullsized":"2024-09-13T17:42:50.731541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:18.016511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694137+0000","last_change":"2024-09-13T17:42:50.694137+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694137+0000","last_undegraded":"2024-09-13T17:42:50.694137+0000","last_fullsized":"2024-09-13T17:42:50.694137+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717647+0000","last_change":"2024-09-13T17:42:50.714076+0000","last_active":"2024-09-13T17:42:50.717647+0000","last_peered":"2024-09-13T17:42:50.717647+0000","last_clean":"2024-09-13T17:42:50.717647+0000","last_became_active":"2024-09-13T17:42:50.713459+0000","last_became_peered":"2024-09-13T17:42:50.713459+0000","last_unstale":"2024-09-13T17:42:50.717647+0000","last_undegraded":"2024-09-13T17:42:50.717647+0000","last_fullsized":"2024-09-13T17:42:50.717647+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:23.522612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693969+0000","last_change":"2024-09-13T17:42:50.693969+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693969+0000","last_undegraded":"2024-09-13T17:42:50.693969+0000","last_fullsized":"2024-09-13T17:42:50.693969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.730807+0000","last_change":"2024-09-13T17:42:50.729261+0000","last_active":"2024-09-13T17:42:50.730807+0000","last_peered":"2024-09-13T17:42:50.730807+0000","last_clean":"2024-09-13T17:42:50.730807+0000","last_became_active":"2024-09-13T17:42:50.728991+0000","last_became_peered":"2024-09-13T17:42:50.728991+0000","last_unstale":"2024-09-13T17:42:50.730807+0000","last_undegraded":"2024-09-13T17:42:50.730807+0000","last_fullsized":"2024-09-13T17:42:50.730807+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:43.188314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694860+0000","last_change":"2024-09-13T17:42:50.694860+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694860+0000","last_undegraded":"2024-09-13T17:42:50.694860+0000","last_fullsized":"2024-09-13T17:42:50.694860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728549+0000","last_change":"2024-09-13T17:42:50.711610+0000","last_active":"2024-09-13T17:42:50.728549+0000","last_peered":"2024-09-13T17:42:50.728549+0000","last_clean":"2024-09-13T17:42:50.728549+0000","last_became_active":"2024-09-13T17:42:50.711348+0000","last_became_peered":"2024-09-13T17:42:50.711348+0000","last_unstale":"2024-09-13T17:42:50.728549+0000","last_undegraded":"2024-09-13T17:42:50.728549+0000","last_fullsized":"2024-09-13T17:42:50.728549+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:41.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692348+0000","last_change":"2024-09-13T17:42:50.692348+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692348+0000","last_undegraded":"2024-09-13T17:42:50.692348+0000","last_fullsized":"2024-09-13T17:42:50.692348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731476+0000","last_change":"2024-09-13T17:42:50.729302+0000","last_active":"2024-09-13T17:42:50.731476+0000","last_peered":"2024-09-13T17:42:50.731476+0000","last_clean":"2024-09-13T17:42:50.731476+0000","last_became_active":"2024-09-13T17:42:50.728824+0000","last_became_peered":"2024-09-13T17:42:50.728824+0000","last_unstale":"2024-09-13T17:42:50.731476+0000","last_undegraded":"2024-09-13T17:42:50.731476+0000","last_fullsized":"2024-09-13T17:42:50.731476+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:15:18.318656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693342+0000","last_change":"2024-09-13T17:42:50.693342+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693342+0000","last_undegraded":"2024-09-13T17:42:50.693342+0000","last_fullsized":"2024-09-13T17:42:50.693342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729236+0000","last_change":"2024-09-13T17:42:50.719066+0000","last_active":"2024-09-13T17:42:50.729236+0000","last_peered":"2024-09-13T17:42:50.729236+0000","last_clean":"2024-09-13T17:42:50.729236+0000","last_became_active":"2024-09-13T17:42:50.717841+0000","last_became_peered":"2024-09-13T17:42:50.717841+0000","last_unstale":"2024-09-13T17:42:50.729236+0000","last_undegraded":"2024-09-13T17:42:50.729236+0000","last_fullsized":"2024-09-13T17:42:50.729236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.692083+0000","last_change":"2024-09-13T17:42:50.692083+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.692083+0000","last_undegraded":"2024-09-13T17:42:50.692083+0000","last_fullsized":"2024-09-13T17:42:50.692083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.721315+0000","last_change":"2024-09-13T17:42:50.715306+0000","last_active":"2024-09-13T17:42:50.721315+0000","last_peered":"2024-09-13T17:42:50.721315+0000","last_clean":"2024-09-13T17:42:50.721315+0000","last_became_active":"2024-09-13T17:42:50.714696+0000","last_became_peered":"2024-09-13T17:42:50.714696+0000","last_unstale":"2024-09-13T17:42:50.721315+0000","last_undegraded":"2024-09-13T17:42:50.721315+0000","last_fullsized":"2024-09-13T17:42:50.721315+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693304+0000","last_change":"2024-09-13T17:42:50.693304+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693304+0000","last_undegraded":"2024-09-13T17:42:50.693304+0000","last_fullsized":"2024-09-13T17:42:50.693304+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728412+0000","last_change":"2024-09-13T17:42:50.707367+0000","last_active":"2024-09-13T17:42:50.728412+0000","last_peered":"2024-09-13T17:42:50.728412+0000","last_clean":"2024-09-13T17:42:50.728412+0000","last_became_active":"2024-09-13T17:42:50.706061+0000","last_became_peered":"2024-09-13T17:42:50.706061+0000","last_unstale":"2024-09-13T17:42:50.728412+0000","last_undegraded":"2024-09-13T17:42:50.728412+0000","last_fullsized":"2024-09-13T17:42:50.728412+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:05.864527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696713+0000","last_change":"2024-09-13T17:42:50.696713+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696713+0000","last_undegraded":"2024-09-13T17:42:50.696713+0000","last_fullsized":"2024-09-13T17:42:50.696713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728606+0000","last_change":"2024-09-13T17:42:50.712051+0000","last_active":"2024-09-13T17:42:50.728606+0000","last_peered":"2024-09-13T17:42:50.728606+0000","last_clean":"2024-09-13T17:42:50.728606+0000","last_became_active":"2024-09-13T17:42:50.711326+0000","last_became_peered":"2024-09-13T17:42:50.711326+0000","last_unstale":"2024-09-13T17:42:50.728606+0000","last_undegraded":"2024-09-13T17:42:50.728606+0000","last_fullsized":"2024-09-13T17:42:50.728606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693033+0000","last_change":"2024-09-13T17:42:50.693033+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693033+0000","last_undegraded":"2024-09-13T17:42:50.693033+0000","last_fullsized":"2024-09-13T17:42:50.693033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.732048+0000","last_change":"2024-09-13T17:42:50.730529+0000","last_active":"2024-09-13T17:42:50.732048+0000","last_peered":"2024-09-13T17:42:50.732048+0000","last_clean":"2024-09-13T17:42:50.732048+0000","last_became_active":"2024-09-13T17:42:50.730251+0000","last_became_peered":"2024-09-13T17:42:50.730251+0000","last_unstale":"2024-09-13T17:42:50.732048+0000","last_undegraded":"2024-09-13T17:42:50.732048+0000","last_fullsized":"2024-09-13T17:42:50.732048+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:31.396994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687292+0000","last_change":"2024-09-13T17:42:50.687292+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687292+0000","last_undegraded":"2024-09-13T17:42:50.687292+0000","last_fullsized":"2024-09-13T17:42:50.687292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729338+0000","last_change":"2024-09-13T17:42:50.716986+0000","last_active":"2024-09-13T17:42:50.729338+0000","last_peered":"2024-09-13T17:42:50.729338+0000","last_clean":"2024-09-13T17:42:50.729338+0000","last_became_active":"2024-09-13T17:42:50.716657+0000","last_became_peered":"2024-09-13T17:42:50.716657+0000","last_unstale":"2024-09-13T17:42:50.729338+0000","last_undegraded":"2024-09-13T17:42:50.729338+0000","last_fullsized":"2024-09-13T17:42:50.729338+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:16.071866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.696247+0000","last_change":"2024-09-13T17:42:50.696247+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.696247+0000","last_undegraded":"2024-09-13T17:42:50.696247+0000","last_fullsized":"2024-09-13T17:42:50.696247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.719978+0000","last_change":"2024-09-13T17:42:50.716335+0000","last_active":"2024-09-13T17:42:50.719978+0000","last_peered":"2024-09-13T17:42:50.719978+0000","last_clean":"2024-09-13T17:42:50.719978+0000","last_became_active":"2024-09-13T17:42:50.716044+0000","last_became_peered":"2024-09-13T17:42:50.716044+0000","last_unstale":"2024-09-13T17:42:50.719978+0000","last_undegraded":"2024-09-13T17:42:50.719978+0000","last_fullsized":"2024-09-13T17:42:50.719978+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:50.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691952+0000","last_change":"2024-09-13T17:42:50.691952+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691952+0000","last_undegraded":"2024-09-13T17:42:50.691952+0000","last_fullsized":"2024-09-13T17:42:50.691952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.723702+0000","last_change":"2024-09-13T17:42:50.717268+0000","last_active":"2024-09-13T17:42:50.723702+0000","last_peered":"2024-09-13T17:42:50.723702+0000","last_clean":"2024-09-13T17:42:50.723702+0000","last_became_active":"2024-09-13T17:42:50.716630+0000","last_became_peered":"2024-09-13T17:42:50.716630+0000","last_unstale":"2024-09-13T17:42:50.723702+0000","last_undegraded":"2024-09-13T17:42:50.723702+0000","last_fullsized":"2024-09-13T17:42:50.723702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:56.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.687270+0000","last_change":"2024-09-13T17:42:50.687270+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.687270+0000","last_undegraded":"2024-09-13T17:42:50.687270+0000","last_fullsized":"2024-09-13T17:42:50.687270+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731410+0000","last_change":"2024-09-13T17:42:50.729119+0000","last_active":"2024-09-13T17:42:50.731410+0000","last_peered":"2024-09-13T17:42:50.731410+0000","last_clean":"2024-09-13T17:42:50.731410+0000","last_became_active":"2024-09-13T17:42:50.728779+0000","last_became_peered":"2024-09-13T17:42:50.728779+0000","last_unstale":"2024-09-13T17:42:50.731410+0000","last_undegraded":"2024-09-13T17:42:50.731410+0000","last_fullsized":"2024-09-13T17:42:50.731410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691690+0000","last_change":"2024-09-13T17:42:50.691690+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691690+0000","last_undegraded":"2024-09-13T17:42:50.691690+0000","last_fullsized":"2024-09-13T17:42:50.691690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729125+0000","last_change":"2024-09-13T17:42:50.714804+0000","last_active":"2024-09-13T17:42:50.729125+0000","last_peered":"2024-09-13T17:42:50.729125+0000","last_clean":"2024-09-13T17:42:50.729125+0000","last_became_active":"2024-09-13T17:42:50.711927+0000","last_became_peered":"2024-09-13T17:42:50.711927+0000","last_unstale":"2024-09-13T17:42:50.729125+0000","last_undegraded":"2024-09-13T17:42:50.729125+0000","last_fullsized":"2024-09-13T17:42:50.729125+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:04.749440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689421+0000","last_change":"2024-09-13T17:42:50.689421+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689421+0000","last_undegraded":"2024-09-13T17:42:50.689421+0000","last_fullsized":"2024-09-13T17:42:50.689421+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729364+0000","last_change":"2024-09-13T17:42:50.716142+0000","last_active":"2024-09-13T17:42:50.729364+0000","last_peered":"2024-09-13T17:42:50.729364+0000","last_clean":"2024-09-13T17:42:50.729364+0000","last_became_active":"2024-09-13T17:42:50.715549+0000","last_became_peered":"2024-09-13T17:42:50.715549+0000","last_unstale":"2024-09-13T17:42:50.729364+0000","last_undegraded":"2024-09-13T17:42:50.729364+0000","last_fullsized":"2024-09-13T17:42:50.729364+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691550+0000","last_change":"2024-09-13T17:42:50.691550+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691550+0000","last_undegraded":"2024-09-13T17:42:50.691550+0000","last_fullsized":"2024-09-13T17:42:50.691550+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.728587+0000","last_change":"2024-09-13T17:42:50.707353+0000","last_active":"2024-09-13T17:42:50.728587+0000","last_peered":"2024-09-13T17:42:50.728587+0000","last_clean":"2024-09-13T17:42:50.728587+0000","last_became_active":"2024-09-13T17:42:50.706452+0000","last_became_peered":"2024-09-13T17:42:50.706452+0000","last_unstale":"2024-09-13T17:42:50.728587+0000","last_undegraded":"2024-09-13T17:42:50.728587+0000","last_fullsized":"2024-09-13T17:42:50.728587+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:15.294714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.694956+0000","last_change":"2024-09-13T17:42:50.694956+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.694956+0000","last_undegraded":"2024-09-13T17:42:50.694956+0000","last_fullsized":"2024-09-13T17:42:50.694956+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.731520+0000","last_change":"2024-09-13T17:42:50.730051+0000","last_active":"2024-09-13T17:42:50.731520+0000","last_peered":"2024-09-13T17:42:50.731520+0000","last_clean":"2024-09-13T17:42:50.731520+0000","last_became_active":"2024-09-13T17:42:50.729726+0000","last_became_peered":"2024-09-13T17:42:50.729726+0000","last_unstale":"2024-09-13T17:42:50.731520+0000","last_undegraded":"2024-09-13T17:42:50.731520+0000","last_fullsized":"2024-09-13T17:42:50.731520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:20.073192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691599+0000","last_change":"2024-09-13T17:42:50.691599+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691599+0000","last_undegraded":"2024-09-13T17:42:50.691599+0000","last_fullsized":"2024-09-13T17:42:50.691599+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729188+0000","last_change":"2024-09-13T17:42:50.717327+0000","last_active":"2024-09-13T17:42:50.729188+0000","last_peered":"2024-09-13T17:42:50.729188+0000","last_clean":"2024-09-13T17:42:50.729188+0000","last_became_active":"2024-09-13T17:42:50.716762+0000","last_became_peered":"2024-09-13T17:42:50.716762+0000","last_unstale":"2024-09-13T17:42:50.729188+0000","last_undegraded":"2024-09-13T17:42:50.729188+0000","last_fullsized":"2024-09-13T17:42:50.729188+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:17.303267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691810+0000","last_change":"2024-09-13T17:42:50.691810+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691810+0000","last_undegraded":"2024-09-13T17:42:50.691810+0000","last_fullsized":"2024-09-13T17:42:50.691810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.717516+0000","last_change":"2024-09-13T17:42:50.711721+0000","last_active":"2024-09-13T17:42:50.717516+0000","last_peered":"2024-09-13T17:42:50.717516+0000","last_clean":"2024-09-13T17:42:50.717516+0000","last_became_active":"2024-09-13T17:42:50.711072+0000","last_became_peered":"2024-09-13T17:42:50.711072+0000","last_unstale":"2024-09-13T17:42:50.717516+0000","last_undegraded":"2024-09-13T17:42:50.717516+0000","last_fullsized":"2024-09-13T17:42:50.717516+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:53.463804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.693573+0000","last_change":"2024-09-13T17:42:50.693573+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.693573+0000","last_undegraded":"2024-09-13T17:42:50.693573+0000","last_fullsized":"2024-09-13T17:42:50.693573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729218+0000","last_change":"2024-09-13T17:42:50.718435+0000","last_active":"2024-09-13T17:42:50.729218+0000","last_peered":"2024-09-13T17:42:50.729218+0000","last_clean":"2024-09-13T17:42:50.729218+0000","last_became_active":"2024-09-13T17:42:50.712419+0000","last_became_peered":"2024-09-13T17:42:50.712419+0000","last_unstale":"2024-09-13T17:42:50.729218+0000","last_undegraded":"2024-09-13T17:42:50.729218+0000","last_fullsized":"2024-09-13T17:42:50.729218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:34.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691245+0000","last_change":"2024-09-13T17:42:50.691245+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691245+0000","last_undegraded":"2024-09-13T17:42:50.691245+0000","last_fullsized":"2024-09-13T17:42:50.691245+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.726179+0000","last_change":"2024-09-13T17:42:50.724850+0000","last_active":"2024-09-13T17:42:50.726179+0000","last_peered":"2024-09-13T17:42:50.726179+0000","last_clean":"2024-09-13T17:42:50.726179+0000","last_became_active":"2024-09-13T17:42:50.724216+0000","last_became_peered":"2024-09-13T17:42:50.724216+0000","last_unstale":"2024-09-13T17:42:50.726179+0000","last_undegraded":"2024-09-13T17:42:50.726179+0000","last_fullsized":"2024-09-13T17:42:50.726179+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.689035+0000","last_change":"2024-09-13T17:42:50.689035+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.689035+0000","last_undegraded":"2024-09-13T17:42:50.689035+0000","last_fullsized":"2024-09-13T17:42:50.689035+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.722805+0000","last_change":"2024-09-13T17:42:50.717580+0000","last_active":"2024-09-13T17:42:50.722805+0000","last_peered":"2024-09-13T17:42:50.722805+0000","last_clean":"2024-09-13T17:42:50.722805+0000","last_became_active":"2024-09-13T17:42:50.716737+0000","last_became_peered":"2024-09-13T17:42:50.716737+0000","last_unstale":"2024-09-13T17:42:50.722805+0000","last_undegraded":"2024-09-13T17:42:50.722805+0000","last_fullsized":"2024-09-13T17:42:50.722805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:44.217628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":23,"state":"creating+peering","last_fresh":"2024-09-13T17:42:50.691862+0000","last_change":"2024-09-13T17:42:50.691862+0000","last_active":"2024-09-13T17:42:50.665253+0000","last_peered":"2024-09-13T17:42:50.665253+0000","last_clean":"2024-09-13T17:42:50.665253+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T17:42:50.691862+0000","last_undegraded":"2024-09-13T17:42:50.691862+0000","last_fullsized":"2024-09-13T17:42:50.691862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-13T17:42:50.729058+0000","last_change":"2024-09-13T17:42:50.712393+0000","last_active":"2024-09-13T17:42:50.729058+0000","last_peered":"2024-09-13T17:42:50.729058+0000","last_clean":"2024-09-13T17:42:50.729058+0000","last_became_active":"2024-09-13T17:42:50.712046+0000","last_became_peered":"2024-09-13T17:42:50.712046+0000","last_unstale":"2024-09-13T17:42:50.729058+0000","last_undegraded":"2024-09-13T17:42:50.729058+0000","last_fullsized":"2024-09-13T17:42:50.729058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":148,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-13T17:42:49.680242+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:49.680242+0000","last_peered":"2024-09-13T17:42:49.680242+0000","last_clean":"2024-09-13T17:42:49.680242+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:49.680242+0000","last_undegraded":"2024-09-13T17:42:49.680242+0000","last_fullsized":"2024-09-13T17:42:49.680242+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738383,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":63322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738384,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":798720,"data_stored":657655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738383,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":653690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:59.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:42:59.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f98019d8640 1 Processor -- start 2024-09-13T17:42:59.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f98019d8640 1 -- start start 2024-09-13T17:42:59.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f98019d8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97fc137490 0x7f97fc139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f98019d8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97fc071670 con 0x7f97fc134670 2024-09-13T17:42:59.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f98019d8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97fc071360 con 0x7f97fc137490 2024-09-13T17:42:59.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f97fa7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97fc137490 0x7f97fc139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f97faffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f97fc134670 0x7f97fc134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44378/0 (socket says 172.21.15.96:44378) 2024-09-13T17:42:59.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.567+0000 7f97faffd640 1 -- 172.21.15.96:0/4154919481 learned_addr learned my addr 172.21.15.96:0/4154919481 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:59.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97fa7fc640 1 -- 172.21.15.96:0/4154919481 >> v1:172.21.15.96:6789/0 conn(0x7f97fc134670 legacy=0x7f97fc134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97fa7fc640 1 -- 172.21.15.96:0/4154919481 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc0a7200 con 0x7f97fc137490 2024-09-13T17:42:59.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97fa7fc640 1 --2- 172.21.15.96:0/4154919481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc137490 0x7f97fc139880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009740 tx=0x7f97e401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d894eab8cf48abc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97f9ffb640 1 -- 172.21.15.96:0/4154919481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97e40ab530 con 0x7f97fc137490 2024-09-13T17:42:59.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97f9ffb640 1 -- 172.21.15.96:0/4154919481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e40a7050 con 0x7f97fc137490 2024-09-13T17:42:59.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f97f9ffb640 1 -- 172.21.15.96:0/4154919481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97e40ab7e0 con 0x7f97fc137490 2024-09-13T17:42:59.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 -- 172.21.15.96:0/4154919481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc137490 msgr2=0x7f97fc139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 --2- 172.21.15.96:0/4154919481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc137490 0x7f97fc139880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009740 tx=0x7f97e401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 -- 172.21.15.96:0/4154919481 shutdown_connections 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 --2- 172.21.15.96:0/4154919481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc137490 0x7f97fc139880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 -- 172.21.15.96:0/4154919481 >> 172.21.15.96:0/4154919481 conn(0x7f97fc098a60 msgr2=0x7f97fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 -- 172.21.15.96:0/4154919481 shutdown_connections 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.571+0000 7f98019d8640 1 -- 172.21.15.96:0/4154919481 wait complete. 2024-09-13T17:42:59.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 Processor -- start 2024-09-13T17:42:59.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 -- start start 2024-09-13T17:42:59.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97fc0a40c0 con 0x7f97fc134670 2024-09-13T17:42:59.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97faffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38962/0 (socket says 172.21.15.96:38962) 2024-09-13T17:42:59.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97faffd640 1 -- 172.21.15.96:0/1865324210 learned_addr learned my addr 172.21.15.96:0/1865324210 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97faffd640 1 -- 172.21.15.96:0/1865324210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4009420 con 0x7f97fc134670 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97faffd640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f97f400cb20 tx=0x7f97f400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f400abf0 con 0x7f97fc134670 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f400ad50 con 0x7f97fc134670 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97fc133b70 con 0x7f97fc134670 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97f4012400 con 0x7f97fc134670 2024-09-13T17:42:59.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97fc0a52e0 con 0x7f97fc134670 2024-09-13T17:42:59.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97fc134a50 con 0x7f97fc134670 2024-09-13T17:42:59.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f97f4012560 con 0x7f97fc134670 2024-09-13T17:42:59.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 0x7f97dc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f97f4047610 con 0x7f97fc134670 2024-09-13T17:42:59.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97fa7fc640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 0x7f97dc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.575+0000 7f97fa7fc640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 0x7f97dc037910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e40ad000 tx=0x7f97e40a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.583+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97f4007070 con 0x7f97fc134670 2024-09-13T17:42:59.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.735+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 --> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97fc09d520 con 0x7f97dc035450 2024-09-13T17:42:59.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.747+0000 7f97f3fff640 1 -- 172.21.15.96:0/1865324210 <== mgr.4099 v2:172.21.15.96:6824/762662892 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249943 (secure 0 0 0) 0x7f97e40ab7a0 con 0x7f97dc035450 2024-09-13T17:42:59.752 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:42:59.753 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:42:59.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 msgr2=0x7f97dc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 0x7f97dc037910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e40ad000 tx=0x7f97e40a6000 comp rx=0 tx=0).stop 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 msgr2=0x7f97fc0a4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f97f400cb20 tx=0x7f97f400a060 comp rx=0 tx=0).stop 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 shutdown_connections 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f97dc035450 0x7f97dc037910 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 --2- 172.21.15.96:0/1865324210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97fc134670 0x7f97fc0a4b20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 >> 172.21.15.96:0/1865324210 conn(0x7f97fc098a60 msgr2=0x7f97fc139220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 shutdown_connections 2024-09-13T17:42:59.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.751+0000 7f98019d8640 1 -- 172.21.15.96:0/1865324210 wait complete. 2024-09-13T17:42:59.769 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-13T17:42:57.809444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":182,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82932,"kb_used_data":1872,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283032588,"statfs":{"total":289910292480,"available":289825370112,"internally_reserved":0,"allocated":1916928,"data_stored":1412806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":82963795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.133001"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085246+0000","last_change":"2024-09-13T17:42:51.083617+0000","last_active":"2024-09-13T17:42:51.085246+0000","last_peered":"2024-09-13T17:42:51.085246+0000","last_clean":"2024-09-13T17:42:51.085246+0000","last_became_active":"2024-09-13T17:42:51.083328+0000","last_became_peered":"2024-09-13T17:42:51.083328+0000","last_unstale":"2024-09-13T17:42:51.085246+0000","last_undegraded":"2024-09-13T17:42:51.085246+0000","last_fullsized":"2024-09-13T17:42:51.085246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.081478+0000","last_change":"2024-09-13T17:42:51.076561+0000","last_active":"2024-09-13T17:42:51.081478+0000","last_peered":"2024-09-13T17:42:51.081478+0000","last_clean":"2024-09-13T17:42:51.081478+0000","last_became_active":"2024-09-13T17:42:51.076244+0000","last_became_peered":"2024-09-13T17:42:51.076244+0000","last_unstale":"2024-09-13T17:42:51.081478+0000","last_undegraded":"2024-09-13T17:42:51.081478+0000","last_fullsized":"2024-09-13T17:42:51.081478+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:57:56.449545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.116295+0000","last_change":"2024-09-13T17:42:51.114105+0000","last_active":"2024-09-13T17:42:51.116295+0000","last_peered":"2024-09-13T17:42:51.116295+0000","last_clean":"2024-09-13T17:42:51.116295+0000","last_became_active":"2024-09-13T17:42:51.113689+0000","last_became_peered":"2024-09-13T17:42:51.113689+0000","last_unstale":"2024-09-13T17:42:51.116295+0000","last_undegraded":"2024-09-13T17:42:51.116295+0000","last_fullsized":"2024-09-13T17:42:51.116295+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:01.046421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.106518+0000","last_change":"2024-09-13T17:42:51.092459+0000","last_active":"2024-09-13T17:42:51.106518+0000","last_peered":"2024-09-13T17:42:51.106518+0000","last_clean":"2024-09-13T17:42:51.106518+0000","last_became_active":"2024-09-13T17:42:51.092007+0000","last_became_peered":"2024-09-13T17:42:51.092007+0000","last_unstale":"2024-09-13T17:42:51.106518+0000","last_undegraded":"2024-09-13T17:42:51.106518+0000","last_fullsized":"2024-09-13T17:42:51.106518+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:12:42.787722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085376+0000","last_change":"2024-09-13T17:42:51.084261+0000","last_active":"2024-09-13T17:42:51.085376+0000","last_peered":"2024-09-13T17:42:51.085376+0000","last_clean":"2024-09-13T17:42:51.085376+0000","last_became_active":"2024-09-13T17:42:51.083924+0000","last_became_peered":"2024-09-13T17:42:51.083924+0000","last_unstale":"2024-09-13T17:42:51.085376+0000","last_undegraded":"2024-09-13T17:42:51.085376+0000","last_fullsized":"2024-09-13T17:42:51.085376+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108662+0000","last_change":"2024-09-13T17:42:51.101993+0000","last_active":"2024-09-13T17:42:51.108662+0000","last_peered":"2024-09-13T17:42:51.108662+0000","last_clean":"2024-09-13T17:42:51.108662+0000","last_became_active":"2024-09-13T17:42:51.101508+0000","last_became_peered":"2024-09-13T17:42:51.101508+0000","last_unstale":"2024-09-13T17:42:51.108662+0000","last_undegraded":"2024-09-13T17:42:51.108662+0000","last_fullsized":"2024-09-13T17:42:51.108662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:31:02.065747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114259+0000","last_change":"2024-09-13T17:42:51.111429+0000","last_active":"2024-09-13T17:42:51.114259+0000","last_peered":"2024-09-13T17:42:51.114259+0000","last_clean":"2024-09-13T17:42:51.114259+0000","last_became_active":"2024-09-13T17:42:51.111144+0000","last_became_peered":"2024-09-13T17:42:51.111144+0000","last_unstale":"2024-09-13T17:42:51.114259+0000","last_undegraded":"2024-09-13T17:42:51.114259+0000","last_fullsized":"2024-09-13T17:42:51.114259+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.105336+0000","last_change":"2024-09-13T17:42:51.103076+0000","last_active":"2024-09-13T17:42:51.105336+0000","last_peered":"2024-09-13T17:42:51.105336+0000","last_clean":"2024-09-13T17:42:51.105336+0000","last_became_active":"2024-09-13T17:42:51.102792+0000","last_became_peered":"2024-09-13T17:42:51.102792+0000","last_unstale":"2024-09-13T17:42:51.105336+0000","last_undegraded":"2024-09-13T17:42:51.105336+0000","last_fullsized":"2024-09-13T17:42:51.105336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.115255+0000","last_change":"2024-09-13T17:42:51.108881+0000","last_active":"2024-09-13T17:42:51.115255+0000","last_peered":"2024-09-13T17:42:51.115255+0000","last_clean":"2024-09-13T17:42:51.115255+0000","last_became_active":"2024-09-13T17:42:51.108651+0000","last_became_peered":"2024-09-13T17:42:51.108651+0000","last_unstale":"2024-09-13T17:42:51.115255+0000","last_undegraded":"2024-09-13T17:42:51.115255+0000","last_fullsized":"2024-09-13T17:42:51.115255+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:40:28.829747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.111534+0000","last_change":"2024-09-13T17:42:51.107210+0000","last_active":"2024-09-13T17:42:51.111534+0000","last_peered":"2024-09-13T17:42:51.111534+0000","last_clean":"2024-09-13T17:42:51.111534+0000","last_became_active":"2024-09-13T17:42:51.106761+0000","last_became_peered":"2024-09-13T17:42:51.106761+0000","last_unstale":"2024-09-13T17:42:51.111534+0000","last_undegraded":"2024-09-13T17:42:51.111534+0000","last_fullsized":"2024-09-13T17:42:51.111534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.104597+0000","last_change":"2024-09-13T17:42:51.101067+0000","last_active":"2024-09-13T17:42:51.104597+0000","last_peered":"2024-09-13T17:42:51.104597+0000","last_clean":"2024-09-13T17:42:51.104597+0000","last_became_active":"2024-09-13T17:42:51.099112+0000","last_became_peered":"2024-09-13T17:42:51.099112+0000","last_unstale":"2024-09-13T17:42:51.104597+0000","last_undegraded":"2024-09-13T17:42:51.104597+0000","last_fullsized":"2024-09-13T17:42:51.104597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:26.015645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.106497+0000","last_change":"2024-09-13T17:42:51.092264+0000","last_active":"2024-09-13T17:42:51.106497+0000","last_peered":"2024-09-13T17:42:51.106497+0000","last_clean":"2024-09-13T17:42:51.106497+0000","last_became_active":"2024-09-13T17:42:51.091671+0000","last_became_peered":"2024-09-13T17:42:51.091671+0000","last_unstale":"2024-09-13T17:42:51.106497+0000","last_undegraded":"2024-09-13T17:42:51.106497+0000","last_fullsized":"2024-09-13T17:42:51.106497+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:55:34.941917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.097856+0000","last_change":"2024-09-13T17:42:51.093146+0000","last_active":"2024-09-13T17:42:51.097856+0000","last_peered":"2024-09-13T17:42:51.097856+0000","last_clean":"2024-09-13T17:42:51.097856+0000","last_became_active":"2024-09-13T17:42:51.092852+0000","last_became_peered":"2024-09-13T17:42:51.092852+0000","last_unstale":"2024-09-13T17:42:51.097856+0000","last_undegraded":"2024-09-13T17:42:51.097856+0000","last_fullsized":"2024-09-13T17:42:51.097856+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085774+0000","last_change":"2024-09-13T17:42:51.083151+0000","last_active":"2024-09-13T17:42:51.085774+0000","last_peered":"2024-09-13T17:42:51.085774+0000","last_clean":"2024-09-13T17:42:51.085774+0000","last_became_active":"2024-09-13T17:42:51.082346+0000","last_became_peered":"2024-09-13T17:42:51.082346+0000","last_unstale":"2024-09-13T17:42:51.085774+0000","last_undegraded":"2024-09-13T17:42:51.085774+0000","last_fullsized":"2024-09-13T17:42:51.085774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:13.272242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.082042+0000","last_change":"2024-09-13T17:42:51.079977+0000","last_active":"2024-09-13T17:42:51.082042+0000","last_peered":"2024-09-13T17:42:51.082042+0000","last_clean":"2024-09-13T17:42:51.082042+0000","last_became_active":"2024-09-13T17:42:51.079043+0000","last_became_peered":"2024-09-13T17:42:51.079043+0000","last_unstale":"2024-09-13T17:42:51.082042+0000","last_undegraded":"2024-09-13T17:42:51.082042+0000","last_fullsized":"2024-09-13T17:42:51.082042+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:56:49.760322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.084054+0000","last_change":"2024-09-13T17:42:51.081578+0000","last_active":"2024-09-13T17:42:51.084054+0000","last_peered":"2024-09-13T17:42:51.084054+0000","last_clean":"2024-09-13T17:42:51.084054+0000","last_became_active":"2024-09-13T17:42:51.080591+0000","last_became_peered":"2024-09-13T17:42:51.080591+0000","last_unstale":"2024-09-13T17:42:51.084054+0000","last_undegraded":"2024-09-13T17:42:51.084054+0000","last_fullsized":"2024-09-13T17:42:51.084054+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:05.734554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085770+0000","last_change":"2024-09-13T17:42:51.082902+0000","last_active":"2024-09-13T17:42:51.085770+0000","last_peered":"2024-09-13T17:42:51.085770+0000","last_clean":"2024-09-13T17:42:51.085770+0000","last_became_active":"2024-09-13T17:42:51.082626+0000","last_became_peered":"2024-09-13T17:42:51.082626+0000","last_unstale":"2024-09-13T17:42:51.085770+0000","last_undegraded":"2024-09-13T17:42:51.085770+0000","last_fullsized":"2024-09-13T17:42:51.085770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:13:25.674551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.083898+0000","last_change":"2024-09-13T17:42:51.080750+0000","last_active":"2024-09-13T17:42:51.083898+0000","last_peered":"2024-09-13T17:42:51.083898+0000","last_clean":"2024-09-13T17:42:51.083898+0000","last_became_active":"2024-09-13T17:42:51.078480+0000","last_became_peered":"2024-09-13T17:42:51.078480+0000","last_unstale":"2024-09-13T17:42:51.083898+0000","last_undegraded":"2024-09-13T17:42:51.083898+0000","last_fullsized":"2024-09-13T17:42:51.083898+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:12:49.305715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.102578+0000","last_change":"2024-09-13T17:42:51.099578+0000","last_active":"2024-09-13T17:42:51.102578+0000","last_peered":"2024-09-13T17:42:51.102578+0000","last_clean":"2024-09-13T17:42:51.102578+0000","last_became_active":"2024-09-13T17:42:51.097903+0000","last_became_peered":"2024-09-13T17:42:51.097903+0000","last_unstale":"2024-09-13T17:42:51.102578+0000","last_undegraded":"2024-09-13T17:42:51.102578+0000","last_fullsized":"2024-09-13T17:42:51.102578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:16:49.884855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085263+0000","last_change":"2024-09-13T17:42:51.083832+0000","last_active":"2024-09-13T17:42:51.085263+0000","last_peered":"2024-09-13T17:42:51.085263+0000","last_clean":"2024-09-13T17:42:51.085263+0000","last_became_active":"2024-09-13T17:42:51.083371+0000","last_became_peered":"2024-09-13T17:42:51.083371+0000","last_unstale":"2024-09-13T17:42:51.085263+0000","last_undegraded":"2024-09-13T17:42:51.085263+0000","last_fullsized":"2024-09-13T17:42:51.085263+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:44.193476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.082003+0000","last_change":"2024-09-13T17:42:51.079223+0000","last_active":"2024-09-13T17:42:51.082003+0000","last_peered":"2024-09-13T17:42:51.082003+0000","last_clean":"2024-09-13T17:42:51.082003+0000","last_became_active":"2024-09-13T17:42:51.078924+0000","last_became_peered":"2024-09-13T17:42:51.078924+0000","last_unstale":"2024-09-13T17:42:51.082003+0000","last_undegraded":"2024-09-13T17:42:51.082003+0000","last_fullsized":"2024-09-13T17:42:51.082003+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:06.777036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089079+0000","last_change":"2024-09-13T17:42:51.084463+0000","last_active":"2024-09-13T17:42:51.089079+0000","last_peered":"2024-09-13T17:42:51.089079+0000","last_clean":"2024-09-13T17:42:51.089079+0000","last_became_active":"2024-09-13T17:42:51.083761+0000","last_became_peered":"2024-09-13T17:42:51.083761+0000","last_unstale":"2024-09-13T17:42:51.089079+0000","last_undegraded":"2024-09-13T17:42:51.089079+0000","last_fullsized":"2024-09-13T17:42:51.089079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.104629+0000","last_change":"2024-09-13T17:42:51.101200+0000","last_active":"2024-09-13T17:42:51.104629+0000","last_peered":"2024-09-13T17:42:51.104629+0000","last_clean":"2024-09-13T17:42:51.104629+0000","last_became_active":"2024-09-13T17:42:51.100946+0000","last_became_peered":"2024-09-13T17:42:51.100946+0000","last_unstale":"2024-09-13T17:42:51.104629+0000","last_undegraded":"2024-09-13T17:42:51.104629+0000","last_fullsized":"2024-09-13T17:42:51.104629+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:50:37.454665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108943+0000","last_change":"2024-09-13T17:42:51.108067+0000","last_active":"2024-09-13T17:42:51.108943+0000","last_peered":"2024-09-13T17:42:51.108943+0000","last_clean":"2024-09-13T17:42:51.108943+0000","last_became_active":"2024-09-13T17:42:51.107641+0000","last_became_peered":"2024-09-13T17:42:51.107641+0000","last_unstale":"2024-09-13T17:42:51.108943+0000","last_undegraded":"2024-09-13T17:42:51.108943+0000","last_fullsized":"2024-09-13T17:42:51.108943+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.115375+0000","last_change":"2024-09-13T17:42:51.111315+0000","last_active":"2024-09-13T17:42:51.115375+0000","last_peered":"2024-09-13T17:42:51.115375+0000","last_clean":"2024-09-13T17:42:51.115375+0000","last_became_active":"2024-09-13T17:42:51.111012+0000","last_became_peered":"2024-09-13T17:42:51.111012+0000","last_unstale":"2024-09-13T17:42:51.115375+0000","last_undegraded":"2024-09-13T17:42:51.115375+0000","last_fullsized":"2024-09-13T17:42:51.115375+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113603+0000","last_change":"2024-09-13T17:42:51.110902+0000","last_active":"2024-09-13T17:42:51.113603+0000","last_peered":"2024-09-13T17:42:51.113603+0000","last_clean":"2024-09-13T17:42:51.113603+0000","last_became_active":"2024-09-13T17:42:51.106950+0000","last_became_peered":"2024-09-13T17:42:51.106950+0000","last_unstale":"2024-09-13T17:42:51.113603+0000","last_undegraded":"2024-09-13T17:42:51.113603+0000","last_fullsized":"2024-09-13T17:42:51.113603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:20:00.861105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113948+0000","last_change":"2024-09-13T17:42:51.111843+0000","last_active":"2024-09-13T17:42:51.113948+0000","last_peered":"2024-09-13T17:42:51.113948+0000","last_clean":"2024-09-13T17:42:51.113948+0000","last_became_active":"2024-09-13T17:42:51.110794+0000","last_became_peered":"2024-09-13T17:42:51.110794+0000","last_unstale":"2024-09-13T17:42:51.113948+0000","last_undegraded":"2024-09-13T17:42:51.113948+0000","last_fullsized":"2024-09-13T17:42:51.113948+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089991+0000","last_change":"2024-09-13T17:42:51.088539+0000","last_active":"2024-09-13T17:42:51.089991+0000","last_peered":"2024-09-13T17:42:51.089991+0000","last_clean":"2024-09-13T17:42:51.089991+0000","last_became_active":"2024-09-13T17:42:51.088078+0000","last_became_peered":"2024-09-13T17:42:51.088078+0000","last_unstale":"2024-09-13T17:42:51.089991+0000","last_undegraded":"2024-09-13T17:42:51.089991+0000","last_fullsized":"2024-09-13T17:42:51.089991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:51.378518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.080348+0000","last_change":"2024-09-13T17:42:51.076524+0000","last_active":"2024-09-13T17:42:51.080348+0000","last_peered":"2024-09-13T17:42:51.080348+0000","last_clean":"2024-09-13T17:42:51.080348+0000","last_became_active":"2024-09-13T17:42:51.076315+0000","last_became_peered":"2024-09-13T17:42:51.076315+0000","last_unstale":"2024-09-13T17:42:51.080348+0000","last_undegraded":"2024-09-13T17:42:51.080348+0000","last_fullsized":"2024-09-13T17:42:51.080348+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:34:26.068444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085745+0000","last_change":"2024-09-13T17:42:51.082542+0000","last_active":"2024-09-13T17:42:51.085745+0000","last_peered":"2024-09-13T17:42:51.085745+0000","last_clean":"2024-09-13T17:42:51.085745+0000","last_became_active":"2024-09-13T17:42:51.082224+0000","last_became_peered":"2024-09-13T17:42:51.082224+0000","last_unstale":"2024-09-13T17:42:51.085745+0000","last_undegraded":"2024-09-13T17:42:51.085745+0000","last_fullsized":"2024-09-13T17:42:51.085745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:57.123570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085196+0000","last_change":"2024-09-13T17:42:51.083145+0000","last_active":"2024-09-13T17:42:51.085196+0000","last_peered":"2024-09-13T17:42:51.085196+0000","last_clean":"2024-09-13T17:42:51.085196+0000","last_became_active":"2024-09-13T17:42:51.082726+0000","last_became_peered":"2024-09-13T17:42:51.082726+0000","last_unstale":"2024-09-13T17:42:51.085196+0000","last_undegraded":"2024-09-13T17:42:51.085196+0000","last_fullsized":"2024-09-13T17:42:51.085196+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.105356+0000","last_change":"2024-09-13T17:42:51.103232+0000","last_active":"2024-09-13T17:42:51.105356+0000","last_peered":"2024-09-13T17:42:51.105356+0000","last_clean":"2024-09-13T17:42:51.105356+0000","last_became_active":"2024-09-13T17:42:51.102852+0000","last_became_peered":"2024-09-13T17:42:51.102852+0000","last_unstale":"2024-09-13T17:42:51.105356+0000","last_undegraded":"2024-09-13T17:42:51.105356+0000","last_fullsized":"2024-09-13T17:42:51.105356+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:06:50.612475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.084661+0000","last_change":"2024-09-13T17:42:51.081940+0000","last_active":"2024-09-13T17:42:51.084661+0000","last_peered":"2024-09-13T17:42:51.084661+0000","last_clean":"2024-09-13T17:42:51.084661+0000","last_became_active":"2024-09-13T17:42:51.081674+0000","last_became_peered":"2024-09-13T17:42:51.081674+0000","last_unstale":"2024-09-13T17:42:51.084661+0000","last_undegraded":"2024-09-13T17:42:51.084661+0000","last_fullsized":"2024-09-13T17:42:51.084661+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:53:42.889611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"24'4","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.115660+0000","last_change":"2024-09-13T17:42:50.729720+0000","last_active":"2024-09-13T17:42:51.115660+0000","last_peered":"2024-09-13T17:42:51.115660+0000","last_clean":"2024-09-13T17:42:51.115660+0000","last_became_active":"2024-09-13T17:42:50.729403+0000","last_became_peered":"2024-09-13T17:42:50.729403+0000","last_unstale":"2024-09-13T17:42:51.115660+0000","last_undegraded":"2024-09-13T17:42:51.115660+0000","last_fullsized":"2024-09-13T17:42:51.115660+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:02.933863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113155+0000","last_change":"2024-09-13T17:42:51.111329+0000","last_active":"2024-09-13T17:42:51.113155+0000","last_peered":"2024-09-13T17:42:51.113155+0000","last_clean":"2024-09-13T17:42:51.113155+0000","last_became_active":"2024-09-13T17:42:51.111078+0000","last_became_peered":"2024-09-13T17:42:51.111078+0000","last_unstale":"2024-09-13T17:42:51.113155+0000","last_undegraded":"2024-09-13T17:42:51.113155+0000","last_fullsized":"2024-09-13T17:42:51.113155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.112962+0000","last_change":"2024-09-13T17:42:50.729650+0000","last_active":"2024-09-13T17:42:51.112962+0000","last_peered":"2024-09-13T17:42:51.112962+0000","last_clean":"2024-09-13T17:42:51.112962+0000","last_became_active":"2024-09-13T17:42:50.729126+0000","last_became_peered":"2024-09-13T17:42:50.729126+0000","last_unstale":"2024-09-13T17:42:51.112962+0000","last_undegraded":"2024-09-13T17:42:51.112962+0000","last_fullsized":"2024-09-13T17:42:51.112962+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:09.600109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089973+0000","last_change":"2024-09-13T17:42:51.088387+0000","last_active":"2024-09-13T17:42:51.089973+0000","last_peered":"2024-09-13T17:42:51.089973+0000","last_clean":"2024-09-13T17:42:51.089973+0000","last_became_active":"2024-09-13T17:42:51.087736+0000","last_became_peered":"2024-09-13T17:42:51.087736+0000","last_unstale":"2024-09-13T17:42:51.089973+0000","last_undegraded":"2024-09-13T17:42:51.089973+0000","last_fullsized":"2024-09-13T17:42:51.089973+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:21.078353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113646+0000","last_change":"2024-09-13T17:42:50.727195+0000","last_active":"2024-09-13T17:42:51.113646+0000","last_peered":"2024-09-13T17:42:51.113646+0000","last_clean":"2024-09-13T17:42:51.113646+0000","last_became_active":"2024-09-13T17:42:50.726846+0000","last_became_peered":"2024-09-13T17:42:50.726846+0000","last_unstale":"2024-09-13T17:42:51.113646+0000","last_undegraded":"2024-09-13T17:42:51.113646+0000","last_fullsized":"2024-09-13T17:42:51.113646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:11.620151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113968+0000","last_change":"2024-09-13T17:42:51.112010+0000","last_active":"2024-09-13T17:42:51.113968+0000","last_peered":"2024-09-13T17:42:51.113968+0000","last_clean":"2024-09-13T17:42:51.113968+0000","last_became_active":"2024-09-13T17:42:51.111090+0000","last_became_peered":"2024-09-13T17:42:51.111090+0000","last_unstale":"2024-09-13T17:42:51.113968+0000","last_undegraded":"2024-09-13T17:42:51.113968+0000","last_fullsized":"2024-09-13T17:42:51.113968+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.107631+0000","last_change":"2024-09-13T17:42:50.730528+0000","last_active":"2024-09-13T17:42:51.107631+0000","last_peered":"2024-09-13T17:42:51.107631+0000","last_clean":"2024-09-13T17:42:51.107631+0000","last_became_active":"2024-09-13T17:42:50.730190+0000","last_became_peered":"2024-09-13T17:42:50.730190+0000","last_unstale":"2024-09-13T17:42:51.107631+0000","last_undegraded":"2024-09-13T17:42:51.107631+0000","last_fullsized":"2024-09-13T17:42:51.107631+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.106609+0000","last_change":"2024-09-13T17:42:51.098984+0000","last_active":"2024-09-13T17:42:51.106609+0000","last_peered":"2024-09-13T17:42:51.106609+0000","last_clean":"2024-09-13T17:42:51.106609+0000","last_became_active":"2024-09-13T17:42:51.092559+0000","last_became_peered":"2024-09-13T17:42:51.092559+0000","last_unstale":"2024-09-13T17:42:51.106609+0000","last_undegraded":"2024-09-13T17:42:51.106609+0000","last_fullsized":"2024-09-13T17:42:51.106609+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:08.856735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114105+0000","last_change":"2024-09-13T17:42:50.729206+0000","last_active":"2024-09-13T17:42:51.114105+0000","last_peered":"2024-09-13T17:42:51.114105+0000","last_clean":"2024-09-13T17:42:51.114105+0000","last_became_active":"2024-09-13T17:42:50.728865+0000","last_became_peered":"2024-09-13T17:42:50.728865+0000","last_unstale":"2024-09-13T17:42:51.114105+0000","last_undegraded":"2024-09-13T17:42:51.114105+0000","last_fullsized":"2024-09-13T17:42:51.114105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:39:09.981862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085336+0000","last_change":"2024-09-13T17:42:51.084098+0000","last_active":"2024-09-13T17:42:51.085336+0000","last_peered":"2024-09-13T17:42:51.085336+0000","last_clean":"2024-09-13T17:42:51.085336+0000","last_became_active":"2024-09-13T17:42:51.082812+0000","last_became_peered":"2024-09-13T17:42:51.082812+0000","last_unstale":"2024-09-13T17:42:51.085336+0000","last_undegraded":"2024-09-13T17:42:51.085336+0000","last_fullsized":"2024-09-13T17:42:51.085336+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.103896+0000","last_change":"2024-09-13T17:42:50.719752+0000","last_active":"2024-09-13T17:42:51.103896+0000","last_peered":"2024-09-13T17:42:51.103896+0000","last_clean":"2024-09-13T17:42:51.103896+0000","last_became_active":"2024-09-13T17:42:50.719113+0000","last_became_peered":"2024-09-13T17:42:50.719113+0000","last_unstale":"2024-09-13T17:42:51.103896+0000","last_undegraded":"2024-09-13T17:42:51.103896+0000","last_fullsized":"2024-09-13T17:42:51.103896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:11:14.661920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113683+0000","last_change":"2024-09-13T17:42:51.101672+0000","last_active":"2024-09-13T17:42:51.113683+0000","last_peered":"2024-09-13T17:42:51.113683+0000","last_clean":"2024-09-13T17:42:51.113683+0000","last_became_active":"2024-09-13T17:42:51.099909+0000","last_became_peered":"2024-09-13T17:42:51.099909+0000","last_unstale":"2024-09-13T17:42:51.113683+0000","last_undegraded":"2024-09-13T17:42:51.113683+0000","last_fullsized":"2024-09-13T17:42:51.113683+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:03.939024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.063707+0000","last_change":"2024-09-13T17:42:50.711236+0000","last_active":"2024-09-13T17:42:51.063707+0000","last_peered":"2024-09-13T17:42:51.063707+0000","last_clean":"2024-09-13T17:42:51.063707+0000","last_became_active":"2024-09-13T17:42:50.710880+0000","last_became_peered":"2024-09-13T17:42:50.710880+0000","last_unstale":"2024-09-13T17:42:51.063707+0000","last_undegraded":"2024-09-13T17:42:51.063707+0000","last_fullsized":"2024-09-13T17:42:51.063707+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:01:20.473381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108603+0000","last_change":"2024-09-13T17:42:51.098792+0000","last_active":"2024-09-13T17:42:51.108603+0000","last_peered":"2024-09-13T17:42:51.108603+0000","last_clean":"2024-09-13T17:42:51.108603+0000","last_became_active":"2024-09-13T17:42:51.096459+0000","last_became_peered":"2024-09-13T17:42:51.096459+0000","last_unstale":"2024-09-13T17:42:51.108603+0000","last_undegraded":"2024-09-13T17:42:51.108603+0000","last_fullsized":"2024-09-13T17:42:51.108603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.071273+0000","last_change":"2024-09-13T17:42:50.729497+0000","last_active":"2024-09-13T17:42:51.071273+0000","last_peered":"2024-09-13T17:42:51.071273+0000","last_clean":"2024-09-13T17:42:51.071273+0000","last_became_active":"2024-09-13T17:42:50.728723+0000","last_became_peered":"2024-09-13T17:42:50.728723+0000","last_unstale":"2024-09-13T17:42:51.071273+0000","last_undegraded":"2024-09-13T17:42:51.071273+0000","last_fullsized":"2024-09-13T17:42:51.071273+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:38:36.241145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.109363+0000","last_change":"2024-09-13T17:42:51.104092+0000","last_active":"2024-09-13T17:42:51.109363+0000","last_peered":"2024-09-13T17:42:51.109363+0000","last_clean":"2024-09-13T17:42:51.109363+0000","last_became_active":"2024-09-13T17:42:51.103833+0000","last_became_peered":"2024-09-13T17:42:51.103833+0000","last_unstale":"2024-09-13T17:42:51.109363+0000","last_undegraded":"2024-09-13T17:42:51.109363+0000","last_fullsized":"2024-09-13T17:42:51.109363+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:57.535944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.067191+0000","last_change":"2024-09-13T17:42:50.729826+0000","last_active":"2024-09-13T17:42:51.067191+0000","last_peered":"2024-09-13T17:42:51.067191+0000","last_clean":"2024-09-13T17:42:51.067191+0000","last_became_active":"2024-09-13T17:42:50.729515+0000","last_became_peered":"2024-09-13T17:42:50.729515+0000","last_unstale":"2024-09-13T17:42:51.067191+0000","last_undegraded":"2024-09-13T17:42:51.067191+0000","last_fullsized":"2024-09-13T17:42:51.067191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:18.016511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114317+0000","last_change":"2024-09-13T17:42:51.111830+0000","last_active":"2024-09-13T17:42:51.114317+0000","last_peered":"2024-09-13T17:42:51.114317+0000","last_clean":"2024-09-13T17:42:51.114317+0000","last_became_active":"2024-09-13T17:42:51.111267+0000","last_became_peered":"2024-09-13T17:42:51.111267+0000","last_unstale":"2024-09-13T17:42:51.114317+0000","last_undegraded":"2024-09-13T17:42:51.114317+0000","last_fullsized":"2024-09-13T17:42:51.114317+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.065546+0000","last_change":"2024-09-13T17:42:50.714076+0000","last_active":"2024-09-13T17:42:51.065546+0000","last_peered":"2024-09-13T17:42:51.065546+0000","last_clean":"2024-09-13T17:42:51.065546+0000","last_became_active":"2024-09-13T17:42:50.713459+0000","last_became_peered":"2024-09-13T17:42:50.713459+0000","last_unstale":"2024-09-13T17:42:51.065546+0000","last_undegraded":"2024-09-13T17:42:51.065546+0000","last_fullsized":"2024-09-13T17:42:51.065546+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:23.522612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.084613+0000","last_change":"2024-09-13T17:42:51.083072+0000","last_active":"2024-09-13T17:42:51.084613+0000","last_peered":"2024-09-13T17:42:51.084613+0000","last_clean":"2024-09-13T17:42:51.084613+0000","last_became_active":"2024-09-13T17:42:51.082354+0000","last_became_peered":"2024-09-13T17:42:51.082354+0000","last_unstale":"2024-09-13T17:42:51.084613+0000","last_undegraded":"2024-09-13T17:42:51.084613+0000","last_fullsized":"2024-09-13T17:42:51.084613+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:14:12.206311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"24'3","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114529+0000","last_change":"2024-09-13T17:42:50.729261+0000","last_active":"2024-09-13T17:42:51.114529+0000","last_peered":"2024-09-13T17:42:51.114529+0000","last_clean":"2024-09-13T17:42:51.114529+0000","last_became_active":"2024-09-13T17:42:50.728991+0000","last_became_peered":"2024-09-13T17:42:50.728991+0000","last_unstale":"2024-09-13T17:42:51.114529+0000","last_undegraded":"2024-09-13T17:42:51.114529+0000","last_fullsized":"2024-09-13T17:42:51.114529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:43.188314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.117195+0000","last_change":"2024-09-13T17:42:51.114098+0000","last_active":"2024-09-13T17:42:51.117195+0000","last_peered":"2024-09-13T17:42:51.117195+0000","last_clean":"2024-09-13T17:42:51.117195+0000","last_became_active":"2024-09-13T17:42:51.113179+0000","last_became_peered":"2024-09-13T17:42:51.113179+0000","last_unstale":"2024-09-13T17:42:51.117195+0000","last_undegraded":"2024-09-13T17:42:51.117195+0000","last_fullsized":"2024-09-13T17:42:51.117195+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:10.397341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"24'6","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.120040+0000","last_change":"2024-09-13T17:42:50.711610+0000","last_active":"2024-09-13T17:42:51.120040+0000","last_peered":"2024-09-13T17:42:51.120040+0000","last_clean":"2024-09-13T17:42:51.120040+0000","last_became_active":"2024-09-13T17:42:50.711348+0000","last_became_peered":"2024-09-13T17:42:50.711348+0000","last_unstale":"2024-09-13T17:42:51.120040+0000","last_undegraded":"2024-09-13T17:42:51.120040+0000","last_fullsized":"2024-09-13T17:42:51.120040+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:41.500208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108680+0000","last_change":"2024-09-13T17:42:51.104087+0000","last_active":"2024-09-13T17:42:51.108680+0000","last_peered":"2024-09-13T17:42:51.108680+0000","last_clean":"2024-09-13T17:42:51.108680+0000","last_became_active":"2024-09-13T17:42:51.103781+0000","last_became_peered":"2024-09-13T17:42:51.103781+0000","last_unstale":"2024-09-13T17:42:51.108680+0000","last_undegraded":"2024-09-13T17:42:51.108680+0000","last_fullsized":"2024-09-13T17:42:51.108680+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114230+0000","last_change":"2024-09-13T17:42:50.729302+0000","last_active":"2024-09-13T17:42:51.114230+0000","last_peered":"2024-09-13T17:42:51.114230+0000","last_clean":"2024-09-13T17:42:51.114230+0000","last_became_active":"2024-09-13T17:42:50.728824+0000","last_became_peered":"2024-09-13T17:42:50.728824+0000","last_unstale":"2024-09-13T17:42:51.114230+0000","last_undegraded":"2024-09-13T17:42:51.114230+0000","last_fullsized":"2024-09-13T17:42:51.114230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:15:18.318656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.101794+0000","last_change":"2024-09-13T17:42:51.087373+0000","last_active":"2024-09-13T17:42:51.101794+0000","last_peered":"2024-09-13T17:42:51.101794+0000","last_clean":"2024-09-13T17:42:51.101794+0000","last_became_active":"2024-09-13T17:42:51.086588+0000","last_became_peered":"2024-09-13T17:42:51.086588+0000","last_unstale":"2024-09-13T17:42:51.101794+0000","last_undegraded":"2024-09-13T17:42:51.101794+0000","last_fullsized":"2024-09-13T17:42:51.101794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:20:02.445387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.078373+0000","last_change":"2024-09-13T17:42:50.719066+0000","last_active":"2024-09-13T17:42:51.078373+0000","last_peered":"2024-09-13T17:42:51.078373+0000","last_clean":"2024-09-13T17:42:51.078373+0000","last_became_active":"2024-09-13T17:42:50.717841+0000","last_became_peered":"2024-09-13T17:42:50.717841+0000","last_unstale":"2024-09-13T17:42:51.078373+0000","last_undegraded":"2024-09-13T17:42:51.078373+0000","last_fullsized":"2024-09-13T17:42:51.078373+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:44:12.775235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.105372+0000","last_change":"2024-09-13T17:42:51.103334+0000","last_active":"2024-09-13T17:42:51.105372+0000","last_peered":"2024-09-13T17:42:51.105372+0000","last_clean":"2024-09-13T17:42:51.105372+0000","last_became_active":"2024-09-13T17:42:51.103116+0000","last_became_peered":"2024-09-13T17:42:51.103116+0000","last_unstale":"2024-09-13T17:42:51.105372+0000","last_undegraded":"2024-09-13T17:42:51.105372+0000","last_fullsized":"2024-09-13T17:42:51.105372+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:26.848843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.071217+0000","last_change":"2024-09-13T17:42:50.715306+0000","last_active":"2024-09-13T17:42:51.071217+0000","last_peered":"2024-09-13T17:42:51.071217+0000","last_clean":"2024-09-13T17:42:51.071217+0000","last_became_active":"2024-09-13T17:42:50.714696+0000","last_became_peered":"2024-09-13T17:42:50.714696+0000","last_unstale":"2024-09-13T17:42:51.071217+0000","last_undegraded":"2024-09-13T17:42:51.071217+0000","last_fullsized":"2024-09-13T17:42:51.071217+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108625+0000","last_change":"2024-09-13T17:42:51.102063+0000","last_active":"2024-09-13T17:42:51.108625+0000","last_peered":"2024-09-13T17:42:51.108625+0000","last_clean":"2024-09-13T17:42:51.108625+0000","last_became_active":"2024-09-13T17:42:51.101571+0000","last_became_peered":"2024-09-13T17:42:51.101571+0000","last_unstale":"2024-09-13T17:42:51.108625+0000","last_undegraded":"2024-09-13T17:42:51.108625+0000","last_fullsized":"2024-09-13T17:42:51.108625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:12:10.605271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"24'8","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.119885+0000","last_change":"2024-09-13T17:42:50.707367+0000","last_active":"2024-09-13T17:42:51.119885+0000","last_peered":"2024-09-13T17:42:51.119885+0000","last_clean":"2024-09-13T17:42:51.119885+0000","last_became_active":"2024-09-13T17:42:50.706061+0000","last_became_peered":"2024-09-13T17:42:50.706061+0000","last_unstale":"2024-09-13T17:42:51.119885+0000","last_undegraded":"2024-09-13T17:42:51.119885+0000","last_fullsized":"2024-09-13T17:42:51.119885+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:05.864527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.090101+0000","last_change":"2024-09-13T17:42:51.087543+0000","last_active":"2024-09-13T17:42:51.090101+0000","last_peered":"2024-09-13T17:42:51.090101+0000","last_clean":"2024-09-13T17:42:51.090101+0000","last_became_active":"2024-09-13T17:42:51.087272+0000","last_became_peered":"2024-09-13T17:42:51.087272+0000","last_unstale":"2024-09-13T17:42:51.090101+0000","last_undegraded":"2024-09-13T17:42:51.090101+0000","last_fullsized":"2024-09-13T17:42:51.090101+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:45.222790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.116220+0000","last_change":"2024-09-13T17:42:50.712051+0000","last_active":"2024-09-13T17:42:51.116220+0000","last_peered":"2024-09-13T17:42:51.116220+0000","last_clean":"2024-09-13T17:42:51.116220+0000","last_became_active":"2024-09-13T17:42:50.711326+0000","last_became_peered":"2024-09-13T17:42:50.711326+0000","last_unstale":"2024-09-13T17:42:51.116220+0000","last_undegraded":"2024-09-13T17:42:51.116220+0000","last_fullsized":"2024-09-13T17:42:51.116220+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.108208+0000","last_change":"2024-09-13T17:42:51.087739+0000","last_active":"2024-09-13T17:42:51.108208+0000","last_peered":"2024-09-13T17:42:51.108208+0000","last_clean":"2024-09-13T17:42:51.108208+0000","last_became_active":"2024-09-13T17:42:51.087441+0000","last_became_peered":"2024-09-13T17:42:51.087441+0000","last_unstale":"2024-09-13T17:42:51.108208+0000","last_undegraded":"2024-09-13T17:42:51.108208+0000","last_fullsized":"2024-09-13T17:42:51.108208+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:57.809064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.110957+0000","last_change":"2024-09-13T17:42:50.730529+0000","last_active":"2024-09-13T17:42:51.110957+0000","last_peered":"2024-09-13T17:42:51.110957+0000","last_clean":"2024-09-13T17:42:51.110957+0000","last_became_active":"2024-09-13T17:42:50.730251+0000","last_became_peered":"2024-09-13T17:42:50.730251+0000","last_unstale":"2024-09-13T17:42:51.110957+0000","last_undegraded":"2024-09-13T17:42:51.110957+0000","last_fullsized":"2024-09-13T17:42:51.110957+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:28:31.396994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.116025+0000","last_change":"2024-09-13T17:42:51.099889+0000","last_active":"2024-09-13T17:42:51.116025+0000","last_peered":"2024-09-13T17:42:51.116025+0000","last_clean":"2024-09-13T17:42:51.116025+0000","last_became_active":"2024-09-13T17:42:51.099660+0000","last_became_peered":"2024-09-13T17:42:51.099660+0000","last_unstale":"2024-09-13T17:42:51.116025+0000","last_undegraded":"2024-09-13T17:42:51.116025+0000","last_fullsized":"2024-09-13T17:42:51.116025+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:11.974433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"24'1","reported_seq":12,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.119262+0000","last_change":"2024-09-13T17:42:50.716986+0000","last_active":"2024-09-13T17:42:51.119262+0000","last_peered":"2024-09-13T17:42:51.119262+0000","last_clean":"2024-09-13T17:42:51.119262+0000","last_became_active":"2024-09-13T17:42:50.716657+0000","last_became_peered":"2024-09-13T17:42:50.716657+0000","last_unstale":"2024-09-13T17:42:51.119262+0000","last_undegraded":"2024-09-13T17:42:51.119262+0000","last_fullsized":"2024-09-13T17:42:51.119262+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:16.071866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089902+0000","last_change":"2024-09-13T17:42:51.087227+0000","last_active":"2024-09-13T17:42:51.089902+0000","last_peered":"2024-09-13T17:42:51.089902+0000","last_clean":"2024-09-13T17:42:51.089902+0000","last_became_active":"2024-09-13T17:42:51.086893+0000","last_became_peered":"2024-09-13T17:42:51.086893+0000","last_unstale":"2024-09-13T17:42:51.089902+0000","last_undegraded":"2024-09-13T17:42:51.089902+0000","last_fullsized":"2024-09-13T17:42:51.089902+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:41:19.021673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.065054+0000","last_change":"2024-09-13T17:42:50.716335+0000","last_active":"2024-09-13T17:42:51.065054+0000","last_peered":"2024-09-13T17:42:51.065054+0000","last_clean":"2024-09-13T17:42:51.065054+0000","last_became_active":"2024-09-13T17:42:50.716044+0000","last_became_peered":"2024-09-13T17:42:50.716044+0000","last_unstale":"2024-09-13T17:42:51.065054+0000","last_undegraded":"2024-09-13T17:42:51.065054+0000","last_fullsized":"2024-09-13T17:42:51.065054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:50.373356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089912+0000","last_change":"2024-09-13T17:42:51.087413+0000","last_active":"2024-09-13T17:42:51.089912+0000","last_peered":"2024-09-13T17:42:51.089912+0000","last_clean":"2024-09-13T17:42:51.089912+0000","last_became_active":"2024-09-13T17:42:51.086979+0000","last_became_peered":"2024-09-13T17:42:51.086979+0000","last_unstale":"2024-09-13T17:42:51.089912+0000","last_undegraded":"2024-09-13T17:42:51.089912+0000","last_fullsized":"2024-09-13T17:42:51.089912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:12.625312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.114482+0000","last_change":"2024-09-13T17:42:50.717268+0000","last_active":"2024-09-13T17:42:51.114482+0000","last_peered":"2024-09-13T17:42:51.114482+0000","last_clean":"2024-09-13T17:42:51.114482+0000","last_became_active":"2024-09-13T17:42:50.716630+0000","last_became_peered":"2024-09-13T17:42:50.716630+0000","last_unstale":"2024-09-13T17:42:51.114482+0000","last_undegraded":"2024-09-13T17:42:51.114482+0000","last_fullsized":"2024-09-13T17:42:51.114482+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:45:56.530782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085358+0000","last_change":"2024-09-13T17:42:51.084214+0000","last_active":"2024-09-13T17:42:51.085358+0000","last_peered":"2024-09-13T17:42:51.085358+0000","last_clean":"2024-09-13T17:42:51.085358+0000","last_became_active":"2024-09-13T17:42:51.083548+0000","last_became_peered":"2024-09-13T17:42:51.083548+0000","last_unstale":"2024-09-13T17:42:51.085358+0000","last_undegraded":"2024-09-13T17:42:51.085358+0000","last_fullsized":"2024-09-13T17:42:51.085358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:54.468965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.071861+0000","last_change":"2024-09-13T17:42:50.729119+0000","last_active":"2024-09-13T17:42:51.071861+0000","last_peered":"2024-09-13T17:42:51.071861+0000","last_clean":"2024-09-13T17:42:51.071861+0000","last_became_active":"2024-09-13T17:42:50.728779+0000","last_became_peered":"2024-09-13T17:42:50.728779+0000","last_unstale":"2024-09-13T17:42:51.071861+0000","last_undegraded":"2024-09-13T17:42:51.071861+0000","last_fullsized":"2024-09-13T17:42:51.071861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:55:01.973266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.089994+0000","last_change":"2024-09-13T17:42:51.087108+0000","last_active":"2024-09-13T17:42:51.089994+0000","last_peered":"2024-09-13T17:42:51.089994+0000","last_clean":"2024-09-13T17:42:51.089994+0000","last_became_active":"2024-09-13T17:42:51.086855+0000","last_became_peered":"2024-09-13T17:42:51.086855+0000","last_unstale":"2024-09-13T17:42:51.089994+0000","last_undegraded":"2024-09-13T17:42:51.089994+0000","last_fullsized":"2024-09-13T17:42:51.089994+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:46:29.573627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.086961+0000","last_change":"2024-09-13T17:42:50.714804+0000","last_active":"2024-09-13T17:42:51.086961+0000","last_peered":"2024-09-13T17:42:51.086961+0000","last_clean":"2024-09-13T17:42:51.086961+0000","last_became_active":"2024-09-13T17:42:50.711927+0000","last_became_peered":"2024-09-13T17:42:50.711927+0000","last_unstale":"2024-09-13T17:42:51.086961+0000","last_undegraded":"2024-09-13T17:42:51.086961+0000","last_fullsized":"2024-09-13T17:42:51.086961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:04.749440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.113662+0000","last_change":"2024-09-13T17:42:51.110568+0000","last_active":"2024-09-13T17:42:51.113662+0000","last_peered":"2024-09-13T17:42:51.113662+0000","last_clean":"2024-09-13T17:42:51.113662+0000","last_became_active":"2024-09-13T17:42:51.108413+0000","last_became_peered":"2024-09-13T17:42:51.108413+0000","last_unstale":"2024-09-13T17:42:51.113662+0000","last_undegraded":"2024-09-13T17:42:51.113662+0000","last_fullsized":"2024-09-13T17:42:51.113662+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:00.992744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.077369+0000","last_change":"2024-09-13T17:42:50.716142+0000","last_active":"2024-09-13T17:42:51.077369+0000","last_peered":"2024-09-13T17:42:51.077369+0000","last_clean":"2024-09-13T17:42:51.077369+0000","last_became_active":"2024-09-13T17:42:50.715549+0000","last_became_peered":"2024-09-13T17:42:50.715549+0000","last_unstale":"2024-09-13T17:42:51.077369+0000","last_undegraded":"2024-09-13T17:42:51.077369+0000","last_fullsized":"2024-09-13T17:42:51.077369+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.090083+0000","last_change":"2024-09-13T17:42:51.087921+0000","last_active":"2024-09-13T17:42:51.090083+0000","last_peered":"2024-09-13T17:42:51.090083+0000","last_clean":"2024-09-13T17:42:51.090083+0000","last_became_active":"2024-09-13T17:42:51.087633+0000","last_became_peered":"2024-09-13T17:42:51.087633+0000","last_unstale":"2024-09-13T17:42:51.090083+0000","last_undegraded":"2024-09-13T17:42:51.090083+0000","last_fullsized":"2024-09-13T17:42:51.090083+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:32:24.527774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"24'2","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.115493+0000","last_change":"2024-09-13T17:42:50.707353+0000","last_active":"2024-09-13T17:42:51.115493+0000","last_peered":"2024-09-13T17:42:51.115493+0000","last_clean":"2024-09-13T17:42:51.115493+0000","last_became_active":"2024-09-13T17:42:50.706452+0000","last_became_peered":"2024-09-13T17:42:50.706452+0000","last_unstale":"2024-09-13T17:42:51.115493+0000","last_undegraded":"2024-09-13T17:42:51.115493+0000","last_fullsized":"2024-09-13T17:42:51.115493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:27:15.294714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.081867+0000","last_change":"2024-09-13T17:42:51.076963+0000","last_active":"2024-09-13T17:42:51.081867+0000","last_peered":"2024-09-13T17:42:51.081867+0000","last_clean":"2024-09-13T17:42:51.081867+0000","last_became_active":"2024-09-13T17:42:51.075979+0000","last_became_peered":"2024-09-13T17:42:51.075979+0000","last_unstale":"2024-09-13T17:42:51.081867+0000","last_undegraded":"2024-09-13T17:42:51.081867+0000","last_fullsized":"2024-09-13T17:42:51.081867+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:21:55.080780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"25'1","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-13T17:42:52.069450+0000","last_change":"2024-09-13T17:42:50.730051+0000","last_active":"2024-09-13T17:42:52.069450+0000","last_peered":"2024-09-13T17:42:52.069450+0000","last_clean":"2024-09-13T17:42:52.069450+0000","last_became_active":"2024-09-13T17:42:50.729726+0000","last_became_peered":"2024-09-13T17:42:50.729726+0000","last_unstale":"2024-09-13T17:42:52.069450+0000","last_undegraded":"2024-09-13T17:42:52.069450+0000","last_fullsized":"2024-09-13T17:42:52.069450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:11:20.073192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.104616+0000","last_change":"2024-09-13T17:42:51.101161+0000","last_active":"2024-09-13T17:42:51.104616+0000","last_peered":"2024-09-13T17:42:51.104616+0000","last_clean":"2024-09-13T17:42:51.104616+0000","last_became_active":"2024-09-13T17:42:51.098207+0000","last_became_peered":"2024-09-13T17:42:51.098207+0000","last_unstale":"2024-09-13T17:42:51.104616+0000","last_undegraded":"2024-09-13T17:42:51.104616+0000","last_fullsized":"2024-09-13T17:42:51.104616+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:19:11.720361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.078728+0000","last_change":"2024-09-13T17:42:50.717327+0000","last_active":"2024-09-13T17:42:51.078728+0000","last_peered":"2024-09-13T17:42:51.078728+0000","last_clean":"2024-09-13T17:42:51.078728+0000","last_became_active":"2024-09-13T17:42:50.716762+0000","last_became_peered":"2024-09-13T17:42:50.716762+0000","last_unstale":"2024-09-13T17:42:51.078728+0000","last_undegraded":"2024-09-13T17:42:51.078728+0000","last_fullsized":"2024-09-13T17:42:51.078728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:14:17.303267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.085724+0000","last_change":"2024-09-13T17:42:51.081950+0000","last_active":"2024-09-13T17:42:51.085724+0000","last_peered":"2024-09-13T17:42:51.085724+0000","last_clean":"2024-09-13T17:42:51.085724+0000","last_became_active":"2024-09-13T17:42:51.081640+0000","last_became_peered":"2024-09-13T17:42:51.081640+0000","last_unstale":"2024-09-13T17:42:51.085724+0000","last_undegraded":"2024-09-13T17:42:51.085724+0000","last_fullsized":"2024-09-13T17:42:51.085724+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:51:28.767151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.064299+0000","last_change":"2024-09-13T17:42:50.711721+0000","last_active":"2024-09-13T17:42:51.064299+0000","last_peered":"2024-09-13T17:42:51.064299+0000","last_clean":"2024-09-13T17:42:51.064299+0000","last_became_active":"2024-09-13T17:42:50.711072+0000","last_became_peered":"2024-09-13T17:42:50.711072+0000","last_unstale":"2024-09-13T17:42:51.064299+0000","last_undegraded":"2024-09-13T17:42:51.064299+0000","last_fullsized":"2024-09-13T17:42:51.064299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:28:53.463804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.080368+0000","last_change":"2024-09-13T17:42:51.076540+0000","last_active":"2024-09-13T17:42:51.080368+0000","last_peered":"2024-09-13T17:42:51.080368+0000","last_clean":"2024-09-13T17:42:51.080368+0000","last_became_active":"2024-09-13T17:42:51.076276+0000","last_became_peered":"2024-09-13T17:42:51.076276+0000","last_unstale":"2024-09-13T17:42:51.080368+0000","last_undegraded":"2024-09-13T17:42:51.080368+0000","last_fullsized":"2024-09-13T17:42:51.080368+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:43:42.098591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.078430+0000","last_change":"2024-09-13T17:42:50.718435+0000","last_active":"2024-09-13T17:42:51.078430+0000","last_peered":"2024-09-13T17:42:51.078430+0000","last_clean":"2024-09-13T17:42:51.078430+0000","last_became_active":"2024-09-13T17:42:50.712419+0000","last_became_peered":"2024-09-13T17:42:50.712419+0000","last_unstale":"2024-09-13T17:42:51.078430+0000","last_undegraded":"2024-09-13T17:42:51.078430+0000","last_fullsized":"2024-09-13T17:42:51.078430+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:34.577463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.109618+0000","last_change":"2024-09-13T17:42:51.108602+0000","last_active":"2024-09-13T17:42:51.109618+0000","last_peered":"2024-09-13T17:42:51.109618+0000","last_clean":"2024-09-13T17:42:51.109618+0000","last_became_active":"2024-09-13T17:42:51.108253+0000","last_became_peered":"2024-09-13T17:42:51.108253+0000","last_unstale":"2024-09-13T17:42:51.109618+0000","last_undegraded":"2024-09-13T17:42:51.109618+0000","last_fullsized":"2024-09-13T17:42:51.109618+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:35:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.071270+0000","last_change":"2024-09-13T17:42:50.724850+0000","last_active":"2024-09-13T17:42:51.071270+0000","last_peered":"2024-09-13T17:42:51.071270+0000","last_clean":"2024-09-13T17:42:51.071270+0000","last_became_active":"2024-09-13T17:42:50.724216+0000","last_became_peered":"2024-09-13T17:42:50.724216+0000","last_unstale":"2024-09-13T17:42:51.071270+0000","last_undegraded":"2024-09-13T17:42:51.071270+0000","last_fullsized":"2024-09-13T17:42:51.071270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:47:40.413916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.111355+0000","last_change":"2024-09-13T17:42:51.106632+0000","last_active":"2024-09-13T17:42:51.111355+0000","last_peered":"2024-09-13T17:42:51.111355+0000","last_clean":"2024-09-13T17:42:51.111355+0000","last_became_active":"2024-09-13T17:42:51.104018+0000","last_became_peered":"2024-09-13T17:42:51.104018+0000","last_unstale":"2024-09-13T17:42:51.111355+0000","last_undegraded":"2024-09-13T17:42:51.111355+0000","last_fullsized":"2024-09-13T17:42:51.111355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:21:03.647423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.063167+0000","last_change":"2024-09-13T17:42:50.717580+0000","last_active":"2024-09-13T17:42:51.063167+0000","last_peered":"2024-09-13T17:42:51.063167+0000","last_clean":"2024-09-13T17:42:51.063167+0000","last_became_active":"2024-09-13T17:42:50.716737+0000","last_became_peered":"2024-09-13T17:42:50.716737+0000","last_unstale":"2024-09-13T17:42:51.063167+0000","last_undegraded":"2024-09-13T17:42:51.063167+0000","last_fullsized":"2024-09-13T17:42:51.063167+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:44.217628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.084677+0000","last_change":"2024-09-13T17:42:51.083236+0000","last_active":"2024-09-13T17:42:51.084677+0000","last_peered":"2024-09-13T17:42:51.084677+0000","last_clean":"2024-09-13T17:42:51.084677+0000","last_became_active":"2024-09-13T17:42:51.082820+0000","last_became_peered":"2024-09-13T17:42:51.082820+0000","last_unstale":"2024-09-13T17:42:51.084677+0000","last_undegraded":"2024-09-13T17:42:51.084677+0000","last_fullsized":"2024-09-13T17:42:51.084677+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:50.665253+0000","last_clean_scrub_stamp":"2024-09-13T17:42:50.665253+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:24:28.053886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.095790+0000","last_change":"2024-09-13T17:42:50.712393+0000","last_active":"2024-09-13T17:42:51.095790+0000","last_peered":"2024-09-13T17:42:51.095790+0000","last_clean":"2024-09-13T17:42:51.095790+0000","last_became_active":"2024-09-13T17:42:50.712046+0000","last_became_peered":"2024-09-13T17:42:50.712046+0000","last_unstale":"2024-09-13T17:42:51.095790+0000","last_undegraded":"2024-09-13T17:42:51.095790+0000","last_fullsized":"2024-09-13T17:42:51.095790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:42:49.660092+0000","last_clean_scrub_stamp":"2024-09-13T17:42:49.660092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:20:16.510836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'97","reported_seq":150,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-09-13T17:42:51.077563+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:42:51.077563+0000","last_peered":"2024-09-13T17:42:51.077563+0000","last_clean":"2024-09-13T17:42:51.077563+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:42:51.077563+0000","last_undegraded":"2024-09-13T17:42:51.077563+0000","last_fullsized":"2024-09-13T17:42:51.077563+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738384,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":233472,"data_stored":75908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.64900000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.62}]}]},{"osd":1,"up_from":8,"seq":34359738385,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":847872,"data_stored":668512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.23899999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.32800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738384,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":835584,"data_stored":668386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:42:59.773 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:42:59.774 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c4bd54640 1 Processor -- start 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c4bd54640 1 -- start start 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c4bd54640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c4409ead0 0x7f4c44139f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c4bd54640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c440a0da0 con 0x7f4c441349f0 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c4bd54640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c440a1040 con 0x7f4c4409ead0 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c49af0640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4c441349f0 0x7f4c44134dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44388/0 (socket says 172.21.15.96:44388) 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c49af0640 1 -- 172.21.15.96:0/993670612 learned_addr learned my addr 172.21.15.96:0/993670612 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:42:59.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.935+0000 7f4c492ef640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c4409ead0 0x7f4c44139f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c492ef640 1 -- 172.21.15.96:0/993670612 >> v1:172.21.15.96:6789/0 conn(0x7f4c441349f0 legacy=0x7f4c44134dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c492ef640 1 -- 172.21.15.96:0/993670612 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c440a12e0 con 0x7f4c4409ead0 2024-09-13T17:42:59.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c492ef640 1 --2- 172.21.15.96:0/993670612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44139f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009740 tx=0x7f4c3401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e64d949c459c959 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c48aee640 1 -- 172.21.15.96:0/993670612 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c340ab530 con 0x7f4c4409ead0 2024-09-13T17:42:59.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c48aee640 1 -- 172.21.15.96:0/993670612 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c340a7050 con 0x7f4c4409ead0 2024-09-13T17:42:59.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c48aee640 1 -- 172.21.15.96:0/993670612 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c340ab7e0 con 0x7f4c4409ead0 2024-09-13T17:42:59.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 -- 172.21.15.96:0/993670612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 msgr2=0x7f4c44139f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/993670612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44139f90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009740 tx=0x7f4c3401c390 comp rx=0 tx=0).stop 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 -- 172.21.15.96:0/993670612 shutdown_connections 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/993670612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44139f90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 -- 172.21.15.96:0/993670612 >> 172.21.15.96:0/993670612 conn(0x7f4c440a7390 msgr2=0x7f4c440a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 -- 172.21.15.96:0/993670612 shutdown_connections 2024-09-13T17:42:59.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.939+0000 7f4c4bd54640 1 -- 172.21.15.96:0/993670612 wait complete. 2024-09-13T17:42:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 Processor -- start 2024-09-13T17:42:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 -- start start 2024-09-13T17:42:59.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4409d9b0 con 0x7f4c4409ead0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c49af0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c49af0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38992/0 (socket says 172.21.15.96:38992) 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c49af0640 1 -- 172.21.15.96:0/815920588 learned_addr learned my addr 172.21.15.96:0/815920588 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c49af0640 1 -- 172.21.15.96:0/815920588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34009420 con 0x7f4c4409ead0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c49af0640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000ca90 tx=0x7f4c4000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c40004ca0 con 0x7f4c4409ead0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40004e00 con 0x7f4c4409ead0 2024-09-13T17:42:59.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c4000aba0 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c44134850 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4409b780 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f4c4000ad00 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4409ae30 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 0x7f4c28037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f4c40047470 con 0x7f4c4409ead0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c492ef640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 0x7f4c28037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:42:59.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.943+0000 7f4c492ef640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 0x7f4c28037910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c340ad000 tx=0x7f4c340a6000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:42:59.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:42:59.947+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c40011b90 con 0x7f4c4409ead0 2024-09-13T17:43:00.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.167+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4c440a5840 con 0x7f4c4409ead0 2024-09-13T17:43:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.171+0000 7f4c3a7fc640 1 -- 172.21.15.96:0/815920588 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4c40018030 con 0x7f4c4409ead0 2024-09-13T17:43:00.175 INFO:teuthology.orchestra.run.smithi096.stderr:updated caps for client.0 2024-09-13T17:43:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 msgr2=0x7f4c28037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 0x7f4c28037910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f4c340ad000 tx=0x7f4c340a6000 comp rx=0 tx=0).stop 2024-09-13T17:43:00.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 msgr2=0x7f4c44098f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000ca90 tx=0x7f4c4000cf50 comp rx=0 tx=0).stop 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 shutdown_connections 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f4c28035450 0x7f4c28037910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 --2- 172.21.15.96:0/815920588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c4409ead0 0x7f4c44098f80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 >> 172.21.15.96:0/815920588 conn(0x7f4c440a7390 msgr2=0x7f4c440a18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 shutdown_connections 2024-09-13T17:43:00.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.175+0000 7f4c4bd54640 1 -- 172.21.15.96:0/815920588 wait complete. 2024-09-13T17:43:00.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:00.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.307+0000 7fcb1578d640 1 Processor -- start 2024-09-13T17:43:00.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.307+0000 7fcb1578d640 1 -- start start 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb1009d690 0x7fcb10138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb10138050 con 0x7fcb101387d0 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb1009c410 con 0x7fcb1009d690 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb1009d690 0x7fcb10138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb1009d690 0x7fcb10138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38994/0 (socket says 172.21.15.96:38994) 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 -- 172.21.15.96:0/7592184 learned_addr learned my addr 172.21.15.96:0/7592184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0e7fc640 1 --1- 172.21.15.96:0/7592184 >> v1:172.21.15.96:6789/0 conn(0x7fcb101387d0 0x7fcb10134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44390/0 (socket says 172.21.15.96:44390) 2024-09-13T17:43:00.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 -- 172.21.15.96:0/7592184 >> v1:172.21.15.96:6789/0 conn(0x7fcb101387d0 legacy=0x7fcb10134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 -- 172.21.15.96:0/7592184 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb1009cba0 con 0x7fcb1009d690 2024-09-13T17:43:00.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0effd640 1 --2- 172.21.15.96:0/7592184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb10138290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb000097f0 tx=0x7fcb0001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5ff66a0cd8ddd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0dffb640 1 -- 172.21.15.96:0/7592184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000a8900 con 0x7fcb1009d690 2024-09-13T17:43:00.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0dffb640 1 -- 172.21.15.96:0/7592184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb000a8a60 con 0x7fcb1009d690 2024-09-13T17:43:00.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb0dffb640 1 -- 172.21.15.96:0/7592184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000a8d20 con 0x7fcb1009d690 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- 172.21.15.96:0/7592184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 msgr2=0x7fcb10138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 --2- 172.21.15.96:0/7592184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb10138290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcb000097f0 tx=0x7fcb0001ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- 172.21.15.96:0/7592184 shutdown_connections 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 --2- 172.21.15.96:0/7592184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb10138290 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- 172.21.15.96:0/7592184 >> 172.21.15.96:0/7592184 conn(0x7fcb1006bc60 msgr2=0x7fcb1006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- 172.21.15.96:0/7592184 shutdown_connections 2024-09-13T17:43:00.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.311+0000 7fcb1578d640 1 -- 172.21.15.96:0/7592184 wait complete. 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 Processor -- start 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 -- start start 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb100a5d10 con 0x7fcb1009d690 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39000/0 (socket says 172.21.15.96:39000) 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0effd640 1 -- 172.21.15.96:0/4041303650 learned_addr learned my addr 172.21.15.96:0/4041303650 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0effd640 1 -- 172.21.15.96:0/4041303650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb000094d0 con 0x7fcb1009d690 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0effd640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcb0001c900 tx=0x7fcb0001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000bf070 con 0x7fcb1009d690 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb000b09b0 con 0x7fcb1009d690 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb100a4470 con 0x7fcb1009d690 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb000b0c70 con 0x7fcb1009d690 2024-09-13T17:43:00.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb1009f490 con 0x7fcb1009d690 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fcb000b7070 con 0x7fcb1009d690 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1009bc00 con 0x7fcb1009d690 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 0x7fcafc037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fcb000e4ff0 con 0x7fcb1009d690 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0e7fc640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 0x7fcafc037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.315+0000 7fcb0e7fc640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 0x7fcafc037910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb10067370 tx=0x7fcb04007a10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.319+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb00116db0 con 0x7fcb1009d690 2024-09-13T17:43:00.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb100a5e50 con 0x7fcb1009d690 2024-09-13T17:43:00.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcaf77fe640 1 -- 172.21.15.96:0/4041303650 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fcb000b5030 con 0x7fcb1009d690 2024-09-13T17:43:00.506 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:00.506 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:43:00.506 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:43:00.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 msgr2=0x7fcafc037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 0x7fcafc037910 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcb10067370 tx=0x7fcb04007a10 comp rx=0 tx=0).stop 2024-09-13T17:43:00.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 msgr2=0x7fcb100a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcb0001c900 tx=0x7fcb0001ba70 comp rx=0 tx=0).stop 2024-09-13T17:43:00.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 shutdown_connections 2024-09-13T17:43:00.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fcafc035450 0x7fcafc037910 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 --2- 172.21.15.96:0/4041303650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb1009d690 0x7fcb100a6a60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 >> 172.21.15.96:0/4041303650 conn(0x7fcb1006bc60 msgr2=0x7fcb10099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 shutdown_connections 2024-09-13T17:43:00.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.503+0000 7fcb1578d640 1 -- 172.21.15.96:0/4041303650 wait complete. 2024-09-13T17:43:00.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:42:51.058239+0000', 'modified': '2024-09-13T17:42:54.067162+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4488}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4488': {'gid': 4488, 'name': 'b', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6829/3530148857', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 3530148857}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 3530148857}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:43:00.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:43:00.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:43:00.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f5560432640 1 Processor -- start 2024-09-13T17:43:00.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f5560432640 1 -- start start 2024-09-13T17:43:00.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f5560432640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f555806f6d0 0x7f5558134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f5560432640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555806e130 con 0x7f555809dc20 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f5560432640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55580a6750 con 0x7f555806f6d0 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f555806f6d0 0x7f5558134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f555806f6d0 0x7f5558134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39014/0 (socket says 172.21.15.96:39014) 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 -- 172.21.15.96:0/2228473094 learned_addr learned my addr 172.21.15.96:0/2228473094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555d9cd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f555809dc20 0x7f5558134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44396/0 (socket says 172.21.15.96:44396) 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 -- 172.21.15.96:0/2228473094 >> v1:172.21.15.96:6789/0 conn(0x7f555809dc20 legacy=0x7f5558134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555d1cc640 1 -- 172.21.15.96:0/2228473094 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953615957 0 0) 0x7f555806e130 con 0x7f555809dc20 2024-09-13T17:43:00.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 -- 172.21.15.96:0/2228473094 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55580712e0 con 0x7f555806f6d0 2024-09-13T17:43:00.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555e1ce640 1 --2- 172.21.15.96:0/2228473094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f5558134ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f55540097f0 tx=0x7f555401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84346b2289fbf2a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555d1cc640 1 -- 172.21.15.96:0/2228473094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540ab400 con 0x7f555806f6d0 2024-09-13T17:43:00.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555d1cc640 1 -- 172.21.15.96:0/2228473094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540a7050 con 0x7f555806f6d0 2024-09-13T17:43:00.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.851+0000 7f555d1cc640 1 -- 172.21.15.96:0/2228473094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540ab6b0 con 0x7f555806f6d0 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/2228473094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 msgr2=0x7f5558134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 --2- 172.21.15.96:0/2228473094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f5558134ab0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f55540097f0 tx=0x7f555401c450 comp rx=0 tx=0).stop 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/2228473094 shutdown_connections 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 --2- 172.21.15.96:0/2228473094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f5558134ab0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/2228473094 >> 172.21.15.96:0/2228473094 conn(0x7f5558098a60 msgr2=0x7f555809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/2228473094 shutdown_connections 2024-09-13T17:43:00.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/2228473094 wait complete. 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 Processor -- start 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- start start 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5558134060 con 0x7f555806f6d0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39026/0 (socket says 172.21.15.96:39026) 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555e1ce640 1 -- 172.21.15.96:0/1848452432 learned_addr learned my addr 172.21.15.96:0/1848452432 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555e1ce640 1 -- 172.21.15.96:0/1848452432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55540094d0 con 0x7f555806f6d0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555e1ce640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f555401a080 tx=0x7f555401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 -- 172.21.15.96:0/1848452432 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540b6420 con 0x7f555806f6d0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 -- 172.21.15.96:0/1848452432 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540b6580 con 0x7f555806f6d0 2024-09-13T17:43:00.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55580a1eb0 con 0x7f555806f6d0 2024-09-13T17:43:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 -- 172.21.15.96:0/1848452432 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540b6840 con 0x7f555806f6d0 2024-09-13T17:43:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555809f790 con 0x7f555806f6d0 2024-09-13T17:43:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 -- 172.21.15.96:0/1848452432 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f55540b2040 con 0x7f555806f6d0 2024-09-13T17:43:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 0x7f553c037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f554effd640 1 -- 172.21.15.96:0/1848452432 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55540e1c10 con 0x7f555806f6d0 2024-09-13T17:43:00.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.855+0000 7f555d9cd640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 0x7f553c037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555d9cd640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 0x7f553c037910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5548007fd0 tx=0x7f5548007920 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 Processor -- start 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- start start 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b42bcce80 con 0x559b42d0b710 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39040/0 (socket says 172.21.15.96:39040) 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555e1ce640 1 -- 172.21.15.96:0/984496120 learned_addr learned my addr 172.21.15.96:0/984496120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555e1ce640 1 -- 172.21.15.96:0/984496120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b4284f890 con 0x559b42d0b710 2024-09-13T17:43:00.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f555e1ce640 1 --2- 172.21.15.96:0/984496120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55540e0a90 tx=0x7f55540c7410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb5962e18594f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f553bfff640 1 -- 172.21.15.96:0/984496120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540cd070 con 0x559b42d0b710 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f553bfff640 1 -- 172.21.15.96:0/984496120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540de070 con 0x559b42d0b710 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f553bfff640 1 -- 172.21.15.96:0/984496120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540bdab0 con 0x559b42d0b710 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- 172.21.15.96:0/984496120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 msgr2=0x559b42d018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 --2- 172.21.15.96:0/984496120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f55540e0a90 tx=0x7f55540c7410 comp rx=0 tx=0).stop 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- 172.21.15.96:0/984496120 shutdown_connections 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 --2- 172.21.15.96:0/984496120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d018f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- 172.21.15.96:0/984496120 >> 172.21.15.96:0/984496120 conn(0x559b42cfc230 msgr2=0x559b42d011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- 172.21.15.96:0/984496120 shutdown_connections 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- 172.21.15.96:0/984496120 wait complete. 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 -- start start 2024-09-13T17:43:00.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.859+0000 7f556357b000 1 Processor -- start 2024-09-13T17:43:00.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b42b5aaf0 con 0x559b42d0b710 2024-09-13T17:43:00.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 10 client.0 _ll_register_callbacks cb 0x559b42cf0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:00.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39046/0 (socket says 172.21.15.96:39046) 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555e1ce640 1 -- 172.21.15.96:0/2792790256 learned_addr learned my addr 172.21.15.96:0/2792790256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555e1ce640 1 -- 172.21.15.96:0/2792790256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555809f790 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555e1ce640 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f55540b37a0 tx=0x7f55540dc070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddbbbf842b499dbc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540cd040 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55540bd860 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b42b105f0 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55540c7990 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 10 client.4577 fetch_fsmap learned FSMap version 14 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x559b42b18ab0 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b42b105f0 con 0x559b42d0b710 2024-09-13T17:43:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 14) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f55540c7bb0 con 0x559b42d0b710 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 10 client.4577 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 10 client.4577 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 20 client.4577 populate_metadata read hostname 'smithi096' 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d11130 0x559b42d37b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 4 client.4577 mds_command: new command op to 4488 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 --> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b42b50790 con 0x559b42d11130 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555d9cd640 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d11130 0x559b42d37b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f555d9cd640 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d11130 0x559b42d37b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=223e70d959cc5026 server_cookie=7079ac9cc681f800 in_seq=0 out_seq=0 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 10 client.4577 ms_handle_connect on v2:172.21.15.96:6828/3530148857 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 1 -- 172.21.15.96:0/2792790256 <== mds.0 v2:172.21.15.96:6828/3530148857 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x559b42b50790 con 0x559b42d11130 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 10 client.4577 handle_command_reply: tid=0 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 client.4577 shutdown 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 msgr2=0x559b42d31760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f55540b37a0 tx=0x7f55540dc070 comp rx=0 tx=0).stop 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 shutdown_connections 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d11130 0x559b42d37b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 --2- 172.21.15.96:0/2792790256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d31760 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f55397fa640 0 client.4577 ms_handle_reset on v2:172.21.15.96:6828/3530148857 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 >> 172.21.15.96:0/2792790256 conn(0x559b42cfc230 msgr2=0x559b42d04490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 shutdown_connections 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 1 -- 172.21.15.96:0/2792790256 wait complete. 2024-09-13T17:43:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.863+0000 7f556357b000 20 client.4577 trim_cache size 0 max 16384 2024-09-13T17:43:00.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f556357b000 1 Processor -- start 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f556357b000 1 -- start start 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f556357b000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f556357b000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b42b580c0 con 0x559b42d0b710 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39048/0 (socket says 172.21.15.96:39048) 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f555e1ce640 1 -- 172.21.15.96:0/780283796 learned_addr learned my addr 172.21.15.96:0/780283796 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f555e1ce640 1 -- 172.21.15.96:0/780283796 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b42afec00 con 0x559b42d0b710 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f555e1ce640 1 --2- 172.21.15.96:0/780283796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f555401c280 tx=0x7f55540d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e4d6e6d41e5096c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f554cff9640 1 -- 172.21.15.96:0/780283796 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540b43f0 con 0x559b42d0b710 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f554cff9640 1 -- 172.21.15.96:0/780283796 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55540b6d20 con 0x559b42d0b710 2024-09-13T17:43:00.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.879+0000 7f554cff9640 1 -- 172.21.15.96:0/780283796 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540bd730 con 0x559b42d0b710 2024-09-13T17:43:00.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/780283796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 msgr2=0x559b42d0a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/780283796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f555401c280 tx=0x7f55540d1000 comp rx=0 tx=0).stop 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/780283796 shutdown_connections 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/780283796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d0a5d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/780283796 >> 172.21.15.96:0/780283796 conn(0x559b42cfc230 msgr2=0x559b42d09ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/780283796 shutdown_connections 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/780283796 wait complete. 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- start start 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 Processor -- start 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b42b539b0 con 0x559b42d0b710 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 10 client.0 _ll_register_callbacks cb 0x559b42b02a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555e1ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39056/0 (socket says 172.21.15.96:39056) 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555e1ce640 1 -- 172.21.15.96:0/3272959134 learned_addr learned my addr 172.21.15.96:0/3272959134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555e1ce640 1 -- 172.21.15.96:0/3272959134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b42b18ab0 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555e1ce640 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55540b6ae0 tx=0x7f55540b6a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a847ee5273c1d527 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55540b43f0 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f55540c7ba0 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b42b23460 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55540b3810 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 10 client.4579 fetch_fsmap learned FSMap version 14 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x559b4284fcc0 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559b42b23460 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 14) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f55540f1020 con 0x559b42d0b710 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 10 client.4579 fetch_fsmap finished waiting for FSMap version 14 2024-09-13T17:43:00.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 10 client.4579 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 20 client.4579 populate_metadata read hostname 'smithi096' 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d6d9e0 0x559b42d6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 4 client.4579 mds_command: new command op to 4488 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 --> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b42b1c9c0 con 0x559b42d6d9e0 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555d9cd640 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d6d9e0 0x559b42d6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 10 client.4579 ms_handle_connect on v2:172.21.15.96:6828/3530148857 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f555d9cd640 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d6d9e0 0x559b42d6ddc0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807618ec90e6892d server_cookie=221fa4818e54783c in_seq=0 out_seq=0 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 1 -- 172.21.15.96:0/3272959134 <== mds.0 v2:172.21.15.96:6828/3530148857 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x559b42b1c9c0 con 0x559b42d6d9e0 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 10 client.4579 handle_command_reply: tid=0 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 client.4579 shutdown 2024-09-13T17:43:00.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 msgr2=0x559b42d6d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55540b6ae0 tx=0x7f55540b6a50 comp rx=0 tx=0).stop 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 shutdown_connections 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:6828/3530148857,v1:172.21.15.96:6829/3530148857] conn(0x559b42d6d9e0 0x559b42d6ddc0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 --2- 172.21.15.96:0/3272959134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559b42d0b710 0x559b42d6d4a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 >> 172.21.15.96:0/3272959134 conn(0x559b42cfc230 msgr2=0x559b42d0a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f55397fa640 0 client.4579 ms_handle_reset on v2:172.21.15.96:6828/3530148857 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 shutdown_connections 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 1 -- 172.21.15.96:0/3272959134 wait complete. 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.883+0000 7f556357b000 20 client.4579 trim_cache size 0 max 16384 2024-09-13T17:43:00.893 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "a54aa7b1-da4c-43a3-b94f-1607e77cbcc4", 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4488, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 9.8293885040000006, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 25, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 25, 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 16.423812559000002 2024-09-13T17:43:00.894 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:00.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 msgr2=0x7f553c037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 0x7f553c037910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5548007fd0 tx=0x7f5548007920 comp rx=0 tx=0).stop 2024-09-13T17:43:00.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 msgr2=0x7f555809e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:00.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f555401a080 tx=0x7f555401ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 shutdown_connections 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f553c035450 0x7f553c037910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 --2- 172.21.15.96:0/1848452432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f555806f6d0 0x7f555809e1a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 >> 172.21.15.96:0/1848452432 conn(0x7f5558098a60 msgr2=0x7f555806fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 shutdown_connections 2024-09-13T17:43:00.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:00.887+0000 7f5560432640 1 -- 172.21.15.96:0/1848452432 wait complete. 2024-09-13T17:43:00.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:43:01.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.011+0000 7f0a2f9d3640 1 Processor -- start 2024-09-13T17:43:01.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.011+0000 7f0a2f9d3640 1 -- start start 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a28137cc0 0x7f0a2809d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2809b2f0 con 0x7f0a281380a0 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2809c160 con 0x7f0a28137cc0 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2d76f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a28137cc0 0x7f0a2809d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2cf6e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0a281380a0 0x7f0a28138510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44408/0 (socket says 172.21.15.96:44408) 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2cf6e640 1 -- 172.21.15.96:0/3271008503 learned_addr learned my addr 172.21.15.96:0/3271008503 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2d76f640 1 -- 172.21.15.96:0/3271008503 >> v1:172.21.15.96:6789/0 conn(0x7f0a281380a0 legacy=0x7f0a28138510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2d76f640 1 -- 172.21.15.96:0/3271008503 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2809c6d0 con 0x7f0a28137cc0 2024-09-13T17:43:01.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a1ffff640 1 -- 172.21.15.96:0/3271008503 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613427989 0 0) 0x7f0a2809b2f0 con 0x7f0a281380a0 2024-09-13T17:43:01.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2d76f640 1 --2- 172.21.15.96:0/3271008503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a2809d9b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc522cc83caefdf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:01.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a1ffff640 1 -- 172.21.15.96:0/3271008503 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a200a56d0 con 0x7f0a28137cc0 2024-09-13T17:43:01.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a1ffff640 1 -- 172.21.15.96:0/3271008503 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a200a5830 con 0x7f0a28137cc0 2024-09-13T17:43:01.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a1ffff640 1 -- 172.21.15.96:0/3271008503 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a200a5af0 con 0x7f0a28137cc0 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/3271008503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 msgr2=0x7f0a2809d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/3271008503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a2809d9b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009d90 tx=0x7f0a2001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/3271008503 shutdown_connections 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/3271008503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a2809d9b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.015+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/3271008503 >> 172.21.15.96:0/3271008503 conn(0x7f0a2806bd80 msgr2=0x7f0a2806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/3271008503 shutdown_connections 2024-09-13T17:43:01.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/3271008503 wait complete. 2024-09-13T17:43:01.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 Processor -- start 2024-09-13T17:43:01.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- start start 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a280a3390 con 0x7f0a28137cc0 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2d76f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2d76f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39072/0 (socket says 172.21.15.96:39072) 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2d76f640 1 -- 172.21.15.96:0/1026343600 learned_addr learned my addr 172.21.15.96:0/1026343600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:01.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2d76f640 1 -- 172.21.15.96:0/1026343600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20009a70 con 0x7f0a28137cc0 2024-09-13T17:43:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2d76f640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0a2001a510 tx=0x7f0a20019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a200ad650 con 0x7f0a28137cc0 2024-09-13T17:43:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a200ad7b0 con 0x7f0a28137cc0 2024-09-13T17:43:01.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a280a51d0 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a200ada90 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a280a60e0 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f0a200b4050 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2809d9b0 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 0x7f0a0c037940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f0a200e2940 con 0x7f0a28137cc0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2cf6e640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 0x7f0a0c037940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:01.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.019+0000 7f0a2cf6e640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 0x7f0a0c037940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a18007fd0 tx=0x7f0a18007980 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:01.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.023+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0a200ca940 con 0x7f0a28137cc0 2024-09-13T17:43:01.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.211+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a28065fd0 con 0x7f0a28137cc0 2024-09-13T17:43:01.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a1dffb640 1 -- 172.21.15.96:0/1026343600 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a200c7030 con 0x7f0a28137cc0 2024-09-13T17:43:01.218 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:43:01.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 msgr2=0x7f0a0c037940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 0x7f0a0c037940 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0a18007fd0 tx=0x7f0a18007980 comp rx=0 tx=0).stop 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 msgr2=0x7f0a280a5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0a2001a510 tx=0x7f0a20019680 comp rx=0 tx=0).stop 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 shutdown_connections 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f0a0c035480 0x7f0a0c037940 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 --2- 172.21.15.96:0/1026343600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a28137cc0 0x7f0a280a5ba0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 >> 172.21.15.96:0/1026343600 conn(0x7f0a2806bd80 msgr2=0x7f0a28137370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 shutdown_connections 2024-09-13T17:43:01.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:01.215+0000 7f0a2f9d3640 1 -- 172.21.15.96:0/1026343600 wait complete. 2024-09-13T17:43:01.235 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:43:01.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:43:01.235 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:43:01.235 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:43:01.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:01.236 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:43:01.236 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:43:01.236 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:43:01.236 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:01.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.236 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:43:01.242 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:43:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:43:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:43:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.242 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:43:01.243 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42501sec preferred_lft 42501sec 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:43:01.244 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:43:01.245 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:01.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.246 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:43:01.294 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:43:01.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.295 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:43:01.345 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:43:01.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:43:01.347 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:43:01.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.421 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:43:01.470 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:43:01.470 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:43:01.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:01.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-13T17:43:01.527 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:43:01.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:01.527 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:01.573 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:01.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:01.574 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:01.622 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:01.679 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:01.679 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:01.733 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:01.734 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:01.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.735 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:01.782 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:43:01.782 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:43:01.827 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:01.841 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:01.841 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:01.895 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:01.896 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:01.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:01.896 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:01.900 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-13T17:43:01.895+0000 7f62738f5580 -1 init, newargv = 0x56409b59f910 newargc=15 2024-09-13T17:43:01.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[36148]: starting ceph client 2024-09-13T17:43:01.905 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[36148]: starting fuse 2024-09-13T17:43:03.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:03.022 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:03.022 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:03.081 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:03.083 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:03.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:03.083 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:03.130 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:03.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:03.131 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:03.132 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:03.133 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:03.251 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:03.434 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:03.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:03.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok status 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:43:03.594 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "36148", 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:03.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 192.168.0.1:0/3143210900", 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/3143210900", 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 14, 2024-09-13T17:43:03.596 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 25, 2024-09-13T17:43:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:43:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:43:03.597 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:43:03.597 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:03.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:03.608 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:43:03.608 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:03.614 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:43:03.615 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:03.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:03.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:03.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:43:03.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.831+0000 7f98c5cbb640 1 Processor -- start 2024-09-13T17:43:03.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.831+0000 7f98c5cbb640 1 -- start start 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98c5cbb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f98c0137490 0x7f98c0139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98c5cbb640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c0071670 con 0x7f98c0134670 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98c5cbb640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c0071360 con 0x7f98c0137490 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98b6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f98c0137490 0x7f98c0139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98bf7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f98c0134670 0x7f98c0134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:56138/0 (socket says 172.21.15.96:56138) 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98bf7fe640 1 -- 172.21.15.96:0/1910703421 learned_addr learned my addr 172.21.15.96:0/1910703421 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:03.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98b6ffd640 1 -- 172.21.15.96:0/1910703421 >> v1:172.21.15.96:6789/0 conn(0x7f98c0134670 legacy=0x7f98c0134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:03.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98b6ffd640 1 -- 172.21.15.96:0/1910703421 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00a7200 con 0x7f98c0137490 2024-09-13T17:43:03.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98beffd640 1 -- 172.21.15.96:0/1910703421 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595537268 0 0) 0x7f98c0071670 con 0x7f98c0134670 2024-09-13T17:43:03.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98b6ffd640 1 --2- 172.21.15.96:0/1910703421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0137490 0x7f98c0139880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009740 tx=0x7f98a801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d49a1eb0537584 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:03.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98beffd640 1 -- 172.21.15.96:0/1910703421 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a80ab530 con 0x7f98c0137490 2024-09-13T17:43:03.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98beffd640 1 -- 172.21.15.96:0/1910703421 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a80a7050 con 0x7f98c0137490 2024-09-13T17:43:03.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98beffd640 1 -- 172.21.15.96:0/1910703421 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98a80ab7e0 con 0x7f98c0137490 2024-09-13T17:43:03.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98c5cbb640 1 -- 172.21.15.96:0/1910703421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0137490 msgr2=0x7f98c0139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:03.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.835+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/1910703421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0137490 0x7f98c0139880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009740 tx=0x7f98a801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:03.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/1910703421 shutdown_connections 2024-09-13T17:43:03.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/1910703421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0137490 0x7f98c0139880 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:03.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/1910703421 >> 172.21.15.96:0/1910703421 conn(0x7f98c0098a60 msgr2=0x7f98c009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:03.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/1910703421 shutdown_connections 2024-09-13T17:43:03.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/1910703421 wait complete. 2024-09-13T17:43:03.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 Processor -- start 2024-09-13T17:43:03.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- start start 2024-09-13T17:43:03.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c00a4d80 con 0x7f98c0134670 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35942/0 (socket says 172.21.15.96:35942) 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bf7fe640 1 -- 172.21.15.96:0/3768017680 learned_addr learned my addr 172.21.15.96:0/3768017680 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bf7fe640 1 -- 172.21.15.96:0/3768017680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8009420 con 0x7f98c0134670 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bf7fe640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c420 tx=0x7f98b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98b00074d0 con 0x7f98c0134670 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0007630 con 0x7f98c0134670 2024-09-13T17:43:03.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c00a41b0 con 0x7f98c0134670 2024-09-13T17:43:03.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98b0004960 con 0x7f98c0134670 2024-09-13T17:43:03.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c00a3da0 con 0x7f98c0134670 2024-09-13T17:43:03.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0139400 con 0x7f98c0134670 2024-09-13T17:43:03.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7f98b0004ac0 con 0x7f98c0134670 2024-09-13T17:43:03.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 0x7f98a0037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:03.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f98b0046c40 con 0x7f98c0134670 2024-09-13T17:43:03.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.839+0000 7f98b6ffd640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 0x7f98a0037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:03.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.843+0000 7f98b6ffd640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 0x7f98a0037910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98a80ad000 tx=0x7f98a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:03.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:03.847+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98b001acd0 con 0x7f98c0134670 2024-09-13T17:43:04.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98c0134a50 con 0x7f98c0134670 2024-09-13T17:43:04.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98bcff9640 1 -- 172.21.15.96:0/3768017680 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98b0018020 con 0x7f98c0134670 2024-09-13T17:43:04.043 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 msgr2=0x7f98a0037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 0x7f98a0037910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98a80ad000 tx=0x7f98a8008040 comp rx=0 tx=0).stop 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 msgr2=0x7f98c01cf7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f98b000c420 tx=0x7f98b000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 shutdown_connections 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7f98a0035450 0x7f98a0037910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 --2- 172.21.15.96:0/3768017680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c0134670 0x7f98c01cf7c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:04.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 >> 172.21.15.96:0/3768017680 conn(0x7f98c0098a60 msgr2=0x7f98c006d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:04.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.039+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 shutdown_connections 2024-09-13T17:43:04.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:04.043+0000 7f98c5cbb640 1 -- 172.21.15.96:0/3768017680 wait complete. 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:43:04.061 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:04.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.062 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:43:04.067 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42498sec preferred_lft 42498sec 2024-09-13T17:43:04.068 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:43:04.069 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:04.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.070 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:43:04.119 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:43:04.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.120 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:43:04.171 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:43:04.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.172 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:43:04.172 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:43:04.172 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:43:04.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:43:04.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:43:04.173 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:43:04.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.247 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:43:04.298 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:43:04.298 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:43:04.298 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:04.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.299 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:43:04.355 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:43:04.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:04.355 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:04.402 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:04.402 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:04.402 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:04.449 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:04.507 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:04.507 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:04.564 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:04.565 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:04.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.566 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:04.615 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:04.615 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:43:04.615 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:43:04.659 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:04.673 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:04.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:04.732 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:04.733 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:04.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:04.733 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:04.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:43:04.731+0000 7fe671d1a580 -1 init, newargv = 0x56108a03c640 newargc=15 2024-09-13T17:43:04.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[36255]: starting ceph client 2024-09-13T17:43:04.741 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:04.742 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[36255]: starting fuse 2024-09-13T17:43:05.843 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:05.859 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:05.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:05.913 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:05.915 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:05.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:05.916 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:05.962 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:05.962 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:43:05.963 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:05.963 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:05.964 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:43:05.965 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:06.083 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.36255.asok 2024-09-13T17:43:06.266 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36255.asok 2024-09-13T17:43:06.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:06.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36255.asok status 2024-09-13T17:43:06.426 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:43:06.426 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "36255", 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4591, 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:06.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4591 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 81196373 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:06.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 81196373 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4591 192.168.0.2:0/81196373", 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/81196373", 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 14, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 25, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:43:06.429 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:06.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:06.440 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:43:06.440 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:06.447 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:43:06.447 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:06.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:06.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:06.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-09-13T17:43:06.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.667+0000 7fef6a5c6640 1 Processor -- start 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.667+0000 7fef6a5c6640 1 -- start start 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fef64134e70 0x7fef6409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6409ed60 con 0x7fef64135250 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6409ddc0 con 0x7fef64134e70 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef63fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fef64134e70 0x7fef6409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef5b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fef64135250 0x7fef64135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:56146/0 (socket says 172.21.15.96:56146) 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef5b7fe640 1 -- 172.21.15.96:0/3600557608 learned_addr learned my addr 172.21.15.96:0/3600557608 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef63fff640 1 -- 172.21.15.96:0/3600557608 >> v1:172.21.15.96:6789/0 conn(0x7fef64135250 legacy=0x7fef64135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:06.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef63fff640 1 -- 172.21.15.96:0/3600557608 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef640a68b0 con 0x7fef64134e70 2024-09-13T17:43:06.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef63fff640 1 --2- 172.21.15.96:0/3600557608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef6409e820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef54009740 tx=0x7fef5401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf0145217cd2b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:06.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef637fe640 1 -- 172.21.15.96:0/3600557608 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef540ab530 con 0x7fef64134e70 2024-09-13T17:43:06.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef637fe640 1 -- 172.21.15.96:0/3600557608 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef540a7050 con 0x7fef64134e70 2024-09-13T17:43:06.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef637fe640 1 -- 172.21.15.96:0/3600557608 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef540ab7e0 con 0x7fef64134e70 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- 172.21.15.96:0/3600557608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 msgr2=0x7fef6409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/3600557608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef6409e820 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fef54009740 tx=0x7fef5401c420 comp rx=0 tx=0).stop 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- 172.21.15.96:0/3600557608 shutdown_connections 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/3600557608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef6409e820 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- 172.21.15.96:0/3600557608 >> 172.21.15.96:0/3600557608 conn(0x7fef64098a60 msgr2=0x7fef6409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- 172.21.15.96:0/3600557608 shutdown_connections 2024-09-13T17:43:06.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.671+0000 7fef6a5c6640 1 -- 172.21.15.96:0/3600557608 wait complete. 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 Processor -- start 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 -- start start 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef640a3690 con 0x7fef64134e70 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:06.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef63fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35974/0 (socket says 172.21.15.96:35974) 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef63fff640 1 -- 172.21.15.96:0/4120086804 learned_addr learned my addr 172.21.15.96:0/4120086804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef63fff640 1 -- 172.21.15.96:0/4120086804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef54009420 con 0x7fef64134e70 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef63fff640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef5401a050 tx=0x7fef540191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef540abea0 con 0x7fef64134e70 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef540b4410 con 0x7fef64134e70 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef640a4020 con 0x7fef64134e70 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fef540b46b0 con 0x7fef64134e70 2024-09-13T17:43:06.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef640a3d60 con 0x7fef64134e70 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6412fbf0 con 0x7fef64134e70 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) v1 ==== 43307+0+0 (secure 0 0 0) 0x7fef540b2040 con 0x7fef64134e70 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 0x7fef44037910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fef540e1810 con 0x7fef64134e70 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef5b7fe640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 0x7fef44037910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:06.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.675+0000 7fef5b7fe640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 0x7fef44037910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef4c001610 tx=0x7fef4c007ad0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:06.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.679+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef540bc3b0 con 0x7fef64134e70 2024-09-13T17:43:06.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:06.883+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) v1 -- 0x7fef641368b0 con 0x7fef64134e70 2024-09-13T17:43:07.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.103+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 6) v1 ==== 43316+0+0 (secure 0 0 0) 0x7fef540bb360 con 0x7fef64134e70 2024-09-13T17:43:07.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef617fa640 1 -- 172.21.15.96:0/4120086804 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v6) v1 ==== 84+0+0 (secure 0 0 0) 0x7fef540c5350 con 0x7fef64134e70 2024-09-13T17:43:07.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 msgr2=0x7fef44037910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:07.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 0x7fef44037910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fef4c001610 tx=0x7fef4c007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 msgr2=0x7fef641cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fef5401a050 tx=0x7fef540191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 shutdown_connections 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:6824/762662892,v1:172.21.15.96:6825/762662892] conn(0x7fef44035450 0x7fef44037910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 --2- 172.21.15.96:0/4120086804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef64134e70 0x7fef641cfc50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 >> 172.21.15.96:0/4120086804 conn(0x7fef64098a60 msgr2=0x7fef64098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 shutdown_connections 2024-09-13T17:43:07.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:07.107+0000 7fef6a5c6640 1 -- 172.21.15.96:0/4120086804 wait complete. 2024-09-13T17:43:07.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:07.130 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dump) 2024-09-13T17:43:07.228 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:07.223+0000 7fa9d34d5180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:43:07.598 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:07.595+0000 7fa9d34d5180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:43:07.724 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:07.719+0000 7fa9d34d5180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:43:08.137 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.134+0000 7fa9d34d5180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:08.187 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.182+0000 7fa9d34d5180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:43:08.234 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.230+0000 7fa9d34d5180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:43:08.307 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.302+0000 7fa9d34d5180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:43:08.355 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.350+0000 7fa9d34d5180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:43:08.427 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.422+0000 7fa9d34d5180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:43:08.475 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.470+0000 7fa9d34d5180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:43:08.613 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.606+0000 7fa9d34d5180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:43:08.662 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.658+0000 7fa9d34d5180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:43:08.878 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:43:08.878 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:43:08.878 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:43:08.885 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:08.882+0000 7fa9d34d5180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:43:09.015 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.010+0000 7fa9d34d5180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:09.122 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.118+0000 7fa9d34d5180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:09.182 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.178+0000 7fa9d34d5180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:43:09.312 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.306+0000 7fa9d34d5180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:09.781 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.774+0000 7fa9d34d5180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:43:09.853 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.846+0000 7fa9d34d5180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:43:09.993 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:09.986+0000 7fa9d34d5180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:43:10.065 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.058+0000 7fa9d34d5180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:43:10.346 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.342+0000 7fa9d34d5180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:43:10.399 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.394+0000 7fa9d34d5180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95d24f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95d24f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95d24f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95d24f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95d24f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95a249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95a249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95a249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95a249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:10.487 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:10.482+0000 7fa95a249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:11.441 INFO:teuthology.orchestra.run.smithi096.stdout:{"date": "Fri Sep 13 17:43:11 2024", "client_count": {"total_clients": 0, "fuse": 0, "kclient": 0, "libcephfs": 0}, "filesystems": {"cephfs": {}}} 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:11.455 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:11.456 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:11.457 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:11.457 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:11.457 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:11.457 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:11.457 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:11.534 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:11.718 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:11.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:11.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok mds_sessions 2024-09-13T17:43:11.878 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:11.879 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:11.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:11.881 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:11.882 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:11.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:02.903473+0000", 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:02.903473+0000", 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:11.884 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:11.885 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:11.885 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:11.885 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:11.885 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:12.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:12.892 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dump) 2024-09-13T17:43:12.981 INFO:teuthology.orchestra.run.smithi096.stdout:{"date": "Fri Sep 13 17:43:12 2024", "client_count": {"total_clients": 0, "fuse": 0, "kclient": 0, "libcephfs": 0}, "filesystems": {"cephfs": {}}} 2024-09-13T17:43:12.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:12.991 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:12.992 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:13.053 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:13.241 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:13.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:13.241 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok mds_sessions 2024-09-13T17:43:13.399 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:13.400 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:13.401 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:02.903473+0000", 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:02.903473+0000", 2024-09-13T17:43:13.402 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:13.403 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:14.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:14.414 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dump) 2024-09-13T17:43:14.505 INFO:teuthology.orchestra.run.smithi096.stdout:{"date": "Fri Sep 13 17:43:14 2024", "client_count": {"total_clients": 0, "fuse": 0, "kclient": 0, "libcephfs": 0}, "filesystems": {"cephfs": {}}} 2024-09-13T17:43:14.514 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:14.514 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:14.515 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:14.516 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:14.517 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:14.577 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:14.765 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:14.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:14.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok mds_sessions 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:14.922 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:14.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:02.903473+0000", 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:02.903473+0000", 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:14.924 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:14.925 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:14.925 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:14.925 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:14.925 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:15.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:15.937 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dump) 2024-09-13T17:43:16.024 INFO:teuthology.orchestra.run.smithi096.stdout:{"date": "Fri Sep 13 17:43:16 2024", "client_count": {"total_clients": 0, "fuse": 0, "kclient": 0, "libcephfs": 0}, "filesystems": {"cephfs": {}}} 2024-09-13T17:43:16.032 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:16.032 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:16.032 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:16.032 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:16.032 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:16.033 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:16.034 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:16.096 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:16.284 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:16.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:16.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok mds_sessions 2024-09-13T17:43:16.434 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:16.434 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:16.435 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:16.436 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:02.903473+0000", 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:02.903473+0000", 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:16.437 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:16.438 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:17.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:17.451 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dump) 2024-09-13T17:43:17.541 INFO:teuthology.orchestra.run.smithi096.stdout:{"date": "Fri Sep 13 17:43:17 2024", "client_count": {"total_clients": 4, "fuse": 2, "kclient": 0, "libcephfs": 2}, "filesystems": {"cephfs": {"4585": {"mount_root": "/", "chit": 75.0, "dlease": 0.0, "ofiles": "0", "oicaps": "1", "oinodes": "0", "rtio": 0.0, "raio": 0.0, "rsp": 0.0, "wtio": 0.0, "waio": 0.0, "wsp": 0.0, "rlatavg": 0.0, "rlatsd": 0.0, "wlatavg": 0.0, "wlatsd": 0.0, "mlatavg": 0.59, "mlatsd": 0.12, "mount_point@host/addr": "/home/ubuntu/cephtest/mnt.0@smithi096/192.168.0.1"}, "4591": {"mount_root": "/", "chit": 66.67, "dlease": 0.0, "ofiles": "0", "oicaps": "1", "oinodes": "0", "rtio": 0.0, "raio": 0.0, "rsp": 0.0, "wtio": 0.0, "waio": 0.0, "wsp": 0.0, "rlatavg": 0.0, "rlatsd": 0.0, "wlatavg": 0.0, "wlatsd": 0.0, "mlatavg": 0.4, "mlatsd": 0.0, "mount_point@host/addr": "/home/ubuntu/cephtest/mnt.1@smithi096/192.168.0.2"}, "4609": {"mount_root": "/", "chit": 100.0, "dlease": 0.0, "ofiles": "0", "oicaps": "1", "oinodes": "0", "rtio": 0.0, "raio": 0.0, "rsp": 0.0, "wtio": 0.0, "waio": 0.0, "wsp": 0.0, "rlatavg": 0.0, "rlatsd": 0.0, "wlatavg": 0.0, "wlatsd": 0.0, "mlatavg": 0.56, "mlatsd": 0.0, "mount_point@host/addr": "N/A@smithi096/172.21.15.96"}, "4610": {"mount_root": "/", "chit": 100.0, "dlease": 0.0, "ofiles": "0", "oicaps": "1", "oinodes": "0", "rtio": 0.0, "raio": 0.0, "rsp": 0.0, "wtio": 0.0, "waio": 0.0, "wsp": 0.0, "rlatavg": 0.0, "rlatsd": 0.0, "wlatavg": 0.0, "wlatsd": 0.0, "mlatavg": 0.91, "mlatsd": 0.0, "mount_point@host/addr": "N/A@smithi096/172.21.15.96"}}}} 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:17.547 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:17.548 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.549 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:17.549 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:17.609 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:17.796 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36148.asok 2024-09-13T17:43:17.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:17.797 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36148.asok mds_sessions 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4585, 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4585 2024-09-13T17:43:17.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3143210900 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4585 v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.1:0/3143210900", 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:17.952 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:17.953 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:02.903473+0000", 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:02.903473+0000", 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:17.954 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:17.964 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:17.965 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:43:17.966 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:18.039 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.36255.asok 2024-09-13T17:43:18.223 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.36255.asok 2024-09-13T17:43:18.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:18.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.36255.asok mds_sessions 2024-09-13T17:43:18.380 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4591, 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4591 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 81196373 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:18.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4591 v1:192.168.0.2:0/81196373", 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/81196373", 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6828", 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:18.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6829", 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3530148857 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:44:05.739642+0000", 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:43:05.739642+0000", 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:43:18.383 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:18.384 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:43:18.384 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 14 2024-09-13T17:43:18.384 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:18.394 DEBUG:tasks.cephfs.test_fstop:metrics={'date': 'Fri Sep 13 17:43:17 2024', 'client_count': {'total_clients': 4, 'fuse': 2, 'kclient': 0, 'libcephfs': 2}, 'filesystems': {'cephfs': {'4585': {'mount_root': '/', 'chit': 75.0, 'dlease': 0.0, 'ofiles': '0', 'oicaps': '1', 'oinodes': '0', 'rtio': 0.0, 'raio': 0.0, 'rsp': 0.0, 'wtio': 0.0, 'waio': 0.0, 'wsp': 0.0, 'rlatavg': 0.0, 'rlatsd': 0.0, 'wlatavg': 0.0, 'wlatsd': 0.0, 'mlatavg': 0.59, 'mlatsd': 0.12, 'mount_point@host/addr': '/home/ubuntu/cephtest/mnt.0@smithi096/192.168.0.1'}, '4591': {'mount_root': '/', 'chit': 66.67, 'dlease': 0.0, 'ofiles': '0', 'oicaps': '1', 'oinodes': '0', 'rtio': 0.0, 'raio': 0.0, 'rsp': 0.0, 'wtio': 0.0, 'waio': 0.0, 'wsp': 0.0, 'rlatavg': 0.0, 'rlatsd': 0.0, 'wlatavg': 0.0, 'wlatsd': 0.0, 'mlatavg': 0.4, 'mlatsd': 0.0, 'mount_point@host/addr': '/home/ubuntu/cephtest/mnt.1@smithi096/192.168.0.2'}, '4609': {'mount_root': '/', 'chit': 100.0, 'dlease': 0.0, 'ofiles': '0', 'oicaps': '1', 'oinodes': '0', 'rtio': 0.0, 'raio': 0.0, 'rsp': 0.0, 'wtio': 0.0, 'waio': 0.0, 'wsp': 0.0, 'rlatavg': 0.0, 'rlatsd': 0.0, 'wlatavg': 0.0, 'wlatsd': 0.0, 'mlatavg': 0.56, 'mlatsd': 0.0, 'mount_point@host/addr': 'N/A@smithi096/172.21.15.96'}, '4610': {'mount_root': '/', 'chit': 100.0, 'dlease': 0.0, 'ofiles': '0', 'oicaps': '1', 'oinodes': '0', 'rtio': 0.0, 'raio': 0.0, 'rsp': 0.0, 'wtio': 0.0, 'waio': 0.0, 'wsp': 0.0, 'rlatavg': 0.0, 'rlatsd': 0.0, 'wlatavg': 0.0, 'wlatsd': 0.0, 'mlatavg': 0.91, 'mlatsd': 0.0, 'mount_point@host/addr': 'N/A@smithi096/172.21.15.96'}}}} 2024-09-13T17:43:18.394 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-09-13T17:43:18.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.510+0000 7f7298e0a640 1 Processor -- start 2024-09-13T17:43:18.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 -- start start 2024-09-13T17:43:18.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7294134e70 0x7f729409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:18.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f729409ed60 con 0x7f7294135250 2024-09-13T17:43:18.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f729409ddc0 con 0x7f7294134e70 2024-09-13T17:43:18.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7294134e70 0x7f729409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:18.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7291d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7294135250 0x7f7294135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46950/0 (socket says 172.21.15.96:46950) 2024-09-13T17:43:18.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7291d9b640 1 -- 172.21.15.96:0/134831805 learned_addr learned my addr 172.21.15.96:0/134831805 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:18.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729259c640 1 -- 172.21.15.96:0/134831805 >> v1:172.21.15.96:6789/0 conn(0x7f7294135250 legacy=0x7f7294135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:18.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729259c640 1 -- 172.21.15.96:0/134831805 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940a68b0 con 0x7f7294134e70 2024-09-13T17:43:18.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729259c640 1 --2- 172.21.15.96:0/134831805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409e820 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d90 tx=0x7f728401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f9ef14a19a64cb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:18.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729159a640 1 -- 172.21.15.96:0/134831805 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72840a57b0 con 0x7f7294134e70 2024-09-13T17:43:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729159a640 1 -- 172.21.15.96:0/134831805 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72840a5910 con 0x7f7294134e70 2024-09-13T17:43:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f729159a640 1 -- 172.21.15.96:0/134831805 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72840a5bd0 con 0x7f7294134e70 2024-09-13T17:43:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 -- 172.21.15.96:0/134831805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 msgr2=0x7f729409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:18.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 --2- 172.21.15.96:0/134831805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409e820 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d90 tx=0x7f728401c970 comp rx=0 tx=0).stop 2024-09-13T17:43:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 -- 172.21.15.96:0/134831805 shutdown_connections 2024-09-13T17:43:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.514+0000 7f7298e0a640 1 --2- 172.21.15.96:0/134831805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409e820 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/134831805 >> 172.21.15.96:0/134831805 conn(0x7f7294098a60 msgr2=0x7f729409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/134831805 shutdown_connections 2024-09-13T17:43:18.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/134831805 wait complete. 2024-09-13T17:43:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 Processor -- start 2024-09-13T17:43:18.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- start start 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72940a1dc0 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f729259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f729259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47956/0 (socket says 172.21.15.96:47956) 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f729259c640 1 -- 172.21.15.96:0/1414579676 learned_addr learned my addr 172.21.15.96:0/1414579676 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f729259c640 1 -- 172.21.15.96:0/1414579676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284009a70 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f729259c640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f728401a5a0 tx=0x7f7284019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72840b9790 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72840b98f0 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72940a2a60 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f72840b9bd0 con 0x7f7294134e70 2024-09-13T17:43:18.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f729413ba70 con 0x7f7294134e70 2024-09-13T17:43:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 10) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f72840b9d30 con 0x7f7294134e70 2024-09-13T17:43:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72941368b0 con 0x7f7294134e70 2024-09-13T17:43:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 0x7f7274037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f72840e30c0 con 0x7f7294134e70 2024-09-13T17:43:18.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7291d9b640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 0x7f7274037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:18.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.518+0000 7f7291d9b640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 0x7f7274037a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f727c001610 tx=0x7f727c007ad0 comp rx=0 tx=0).ready entity=mgr.4597 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:18.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.526+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f72840ac3a0 con 0x7f7294134e70 2024-09-13T17:43:18.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:18.718+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) v1 -- 0x7f729412eac0 con 0x7f7294134e70 2024-09-13T17:43:19.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.498+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f72840ad9b0 con 0x7f7294134e70 2024-09-13T17:43:19.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.498+0000 7f728affd640 1 -- 172.21.15.96:0/1414579676 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v11) v1 ==== 85+0+0 (secure 0 0 0) 0x7f72840ace80 con 0x7f7294134e70 2024-09-13T17:43:19.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 msgr2=0x7f7274037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:19.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 0x7f7274037a80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f727c001610 tx=0x7f727c007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 msgr2=0x7f729409f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:19.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f728401a5a0 tx=0x7f7284019710 comp rx=0 tx=0).stop 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 shutdown_connections 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f72740355c0 0x7f7274037a80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 --2- 172.21.15.96:0/1414579676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7294134e70 0x7f729409f280 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 >> 172.21.15.96:0/1414579676 conn(0x7f7294098a60 msgr2=0x7f7294098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 shutdown_connections 2024-09-13T17:43:19.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.502+0000 7f7298e0a640 1 -- 172.21.15.96:0/1414579676 wait complete. 2024-09-13T17:43:19.527 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:43:19.527 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:43:19.607 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:43:19.607 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:19.607 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:19.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:19.628 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:19.622+0000 7fecf5e02180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:43:19.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dce84e640 1 Processor -- start 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dce84e640 1 -- start start 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dce84e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2dc813c840 0x7f2dc813cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dce84e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc806d910 con 0x7f2dc806f250 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dce84e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc806e310 con 0x7f2dc813c840 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2dc813c840 0x7f2dc813cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2dc813c840 0x7f2dc813cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47972/0 (socket says 172.21.15.96:47972) 2024-09-13T17:43:19.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc7fff640 1 -- 172.21.15.96:0/948616505 learned_addr learned my addr 172.21.15.96:0/948616505 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc77fe640 1 --1- 172.21.15.96:0/948616505 >> v1:172.21.15.96:6789/0 conn(0x7f2dc806f250 0x7f2dc80a4900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46960/0 (socket says 172.21.15.96:46960) 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc7fff640 1 -- 172.21.15.96:0/948616505 >> v1:172.21.15.96:6789/0 conn(0x7f2dc806f250 legacy=0x7f2dc80a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.734+0000 7f2dc7fff640 1 -- 172.21.15.96:0/948616505 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80a6680 con 0x7f2dc813c840 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dc7fff640 1 --2- 172.21.15.96:0/948616505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc813c840 0x7f2dc813cc20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a890 tx=0x7f2db802d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70a1954f3e48a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dc6ffd640 1 -- 172.21.15.96:0/948616505 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db80ba540 con 0x7f2dc813c840 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dc6ffd640 1 -- 172.21.15.96:0/948616505 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db80ba6a0 con 0x7f2dc813c840 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dc6ffd640 1 -- 172.21.15.96:0/948616505 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db80ba960 con 0x7f2dc813c840 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- 172.21.15.96:0/948616505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc813c840 msgr2=0x7f2dc813cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 --2- 172.21.15.96:0/948616505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc813c840 0x7f2dc813cc20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a890 tx=0x7f2db802d520 comp rx=0 tx=0).stop 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- 172.21.15.96:0/948616505 shutdown_connections 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 --2- 172.21.15.96:0/948616505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc813c840 0x7f2dc813cc20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- 172.21.15.96:0/948616505 >> 172.21.15.96:0/948616505 conn(0x7f2dc806a150 msgr2=0x7f2dc806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:19.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- 172.21.15.96:0/948616505 shutdown_connections 2024-09-13T17:43:19.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- 172.21.15.96:0/948616505 wait complete. 2024-09-13T17:43:19.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 Processor -- start 2024-09-13T17:43:19.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- start start 2024-09-13T17:43:19.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:19.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dce84e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc813b240 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.738+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47988/0 (socket says 172.21.15.96:47988) 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc7fff640 1 -- 172.21.15.96:0/3411047578 learned_addr learned my addr 172.21.15.96:0/3411047578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc7fff640 1 -- 172.21.15.96:0/3411047578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db801a570 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc7fff640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a890 tx=0x7f2db802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db800b070 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db8011420 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2db80116e0 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc8139740 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc8141f00 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f2db800f070 con 0x7f2dc806f250 2024-09-13T17:43:19.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 0x7f2db4037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:19.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc77fe640 1 -- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 msgr2=0x7f2db4037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:19.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f2db80e2110 con 0x7f2dc806f250 2024-09-13T17:43:19.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dc77fe640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 0x7f2db4037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:19.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.742+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc8073570 con 0x7f2dc806f250 2024-09-13T17:43:19.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.746+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2db8016020 con 0x7f2dc806f250 2024-09-13T17:43:19.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.946+0000 7f2dc77fe640 1 -- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 msgr2=0x7f2db4037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:19.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.946+0000 7f2dc77fe640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 0x7f2db4037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:43:19.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.954+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2dc806ddf0 con 0x7f2dc806f250 2024-09-13T17:43:19.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.958+0000 7f2dc4ff9640 1 -- 172.21.15.96:0/3411047578 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+10866 (secure 0 0 0) 0x7f2db80be070 con 0x7f2dc806f250 2024-09-13T17:43:19.964 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:19.964 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:10.419856+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 msgr2=0x7f2db4037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 0x7f2db4037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 msgr2=0x7f2dc8134bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2db801a890 tx=0x7f2db802a2c0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 shutdown_connections 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f2db4035570 0x7f2db4037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 --2- 172.21.15.96:0/3411047578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dc806f250 0x7f2dc8134bc0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 >> 172.21.15.96:0/3411047578 conn(0x7f2dc806a150 msgr2=0x7f2dc813b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 shutdown_connections 2024-09-13T17:43:19.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:19.962+0000 7f2dce84e640 1 -- 172.21.15.96:0/3411047578 wait complete. 2024-09-13T17:43:19.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:49.224212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:50.030926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.0800000429153442, 'score_stable': 1.0800000429153442, 'optimal_score': 1, 'raw_score_acting': 1.0800000429153442, 'raw_score_stable': 1.0800000429153442, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:19.982 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:20.067 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.062+0000 7fecf5e02180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:43:20.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.090+0000 7f88968ae640 1 Processor -- start 2024-09-13T17:43:20.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.090+0000 7f88968ae640 1 -- start start 2024-09-13T17:43:20.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f88968ae640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8890137360 0x7f889009f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f88968ae640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88900a0310 con 0x7f88900a12a0 2024-09-13T17:43:20.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f88968ae640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88900a0870 con 0x7f8890137360 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8890137360 0x7f889009f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f88900a12a0 0x7f8890137740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46968/0 (socket says 172.21.15.96:46968) 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8890137360 0x7f889009f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48004/0 (socket says 172.21.15.96:48004) 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888f7fe640 1 -- 172.21.15.96:0/3567170211 learned_addr learned my addr 172.21.15.96:0/3567170211 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888ffff640 1 -- 172.21.15.96:0/3567170211 >> v1:172.21.15.96:6789/0 conn(0x7f88900a12a0 legacy=0x7f8890137740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888ffff640 1 -- 172.21.15.96:0/3567170211 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889006b3a0 con 0x7f8890137360 2024-09-13T17:43:20.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888ffff640 1 --2- 172.21.15.96:0/3567170211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8890137360 0x7f889009f700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8888009740 tx=0x7f888801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc980c0cd2363a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888effd640 1 -- 172.21.15.96:0/3567170211 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88880ab530 con 0x7f8890137360 2024-09-13T17:43:20.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888effd640 1 -- 172.21.15.96:0/3567170211 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880a7050 con 0x7f8890137360 2024-09-13T17:43:20.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f888effd640 1 -- 172.21.15.96:0/3567170211 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88880ab7e0 con 0x7f8890137360 2024-09-13T17:43:20.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f88968ae640 1 -- 172.21.15.96:0/3567170211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8890137360 msgr2=0x7f889009f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.094+0000 7f88968ae640 1 --2- 172.21.15.96:0/3567170211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8890137360 0x7f889009f700 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f8888009740 tx=0x7f888801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:20.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/3567170211 shutdown_connections 2024-09-13T17:43:20.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 --2- 172.21.15.96:0/3567170211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8890137360 0x7f889009f700 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/3567170211 >> 172.21.15.96:0/3567170211 conn(0x7f889006f350 msgr2=0x7f8890071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/3567170211 shutdown_connections 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/3567170211 wait complete. 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 Processor -- start 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- start start 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88900a21d0 con 0x7f88900a12a0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48006/0 (socket says 172.21.15.96:48006) 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888ffff640 1 -- 172.21.15.96:0/586477427 learned_addr learned my addr 172.21.15.96:0/586477427 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888ffff640 1 -- 172.21.15.96:0/586477427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888009420 con 0x7f88900a12a0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888ffff640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f88880ae040 tx=0x7f8888019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88880b4540 con 0x7f88900a12a0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88880b46a0 con 0x7f88900a12a0 2024-09-13T17:43:20.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8890135700 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88880b4980 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88900a1c20 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88900a47c0 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f88880b2040 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 0x7f8870037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f88880e1ce0 con 0x7f88900a12a0 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888f7fe640 1 -- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 msgr2=0x7f8870037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:20.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.098+0000 7f888f7fe640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 0x7f8870037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:20.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.106+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f88880b4e60 con 0x7f88900a12a0 2024-09-13T17:43:20.204 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.198+0000 7fecf5e02180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:43:20.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f889009f700 con 0x7f88900a12a0 2024-09-13T17:43:20.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f888cff9640 1 -- 172.21.15.96:0/586477427 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f88880b0030 con 0x7f88900a12a0 2024-09-13T17:43:20.262 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:20.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 msgr2=0x7f8870037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:20.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 0x7f8870037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 msgr2=0x7f889013b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f88880ae040 tx=0x7f8888019130 comp rx=0 tx=0).stop 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 shutdown_connections 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f8870035570 0x7f8870037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 --2- 172.21.15.96:0/586477427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88900a12a0 0x7f889013b6f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 >> 172.21.15.96:0/586477427 conn(0x7f889006f350 msgr2=0x7f88901360f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 shutdown_connections 2024-09-13T17:43:20.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.258+0000 7f88968ae640 1 -- 172.21.15.96:0/586477427 wait complete. 2024-09-13T17:43:20.280 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:20.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00e210c640 1 Processor -- start 2024-09-13T17:43:20.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00e210c640 1 -- start start 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00e210c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f00dc13af50 0x7f00dc13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00e210c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc138050 con 0x7f00dc09d690 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00e210c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00dc09c410 con 0x7f00dc13af50 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00db7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f00dc09d690 0x7f00dc138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46972/0 (socket says 172.21.15.96:46972) 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00db7fe640 1 -- 172.21.15.96:0/3258837083 learned_addr learned my addr 172.21.15.96:0/3258837083 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00daffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f00dc13af50 0x7f00dc13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00daffd640 1 -- 172.21.15.96:0/3258837083 >> v1:172.21.15.96:6789/0 conn(0x7f00dc09d690 legacy=0x7f00dc138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00daffd640 1 -- 172.21.15.96:0/3258837083 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc09cba0 con 0x7f00dc13af50 2024-09-13T17:43:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00daffd640 1 --2- 172.21.15.96:0/3258837083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc13af50 0x7f00dc13d340 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009740 tx=0x7f00d001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ce41462506fc62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00da7fc640 1 -- 172.21.15.96:0/3258837083 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d00a8840 con 0x7f00dc13af50 2024-09-13T17:43:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00da7fc640 1 -- 172.21.15.96:0/3258837083 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d00a89a0 con 0x7f00dc13af50 2024-09-13T17:43:20.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.390+0000 7f00da7fc640 1 -- 172.21.15.96:0/3258837083 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00d00a8c60 con 0x7f00dc13af50 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3258837083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc13af50 msgr2=0x7f00dc13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 --2- 172.21.15.96:0/3258837083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc13af50 0x7f00dc13d340 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009740 tx=0x7f00d001ec10 comp rx=0 tx=0).stop 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3258837083 shutdown_connections 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 --2- 172.21.15.96:0/3258837083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc13af50 0x7f00dc13d340 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3258837083 >> 172.21.15.96:0/3258837083 conn(0x7f00dc06bc60 msgr2=0x7f00dc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3258837083 shutdown_connections 2024-09-13T17:43:20.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3258837083 wait complete. 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 Processor -- start 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- start start 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0a4f90 con 0x7f00dc09d690 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00db7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48034/0 (socket says 172.21.15.96:48034) 2024-09-13T17:43:20.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00db7fe640 1 -- 172.21.15.96:0/3800356069 learned_addr learned my addr 172.21.15.96:0/3800356069 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00db7fe640 1 -- 172.21.15.96:0/3800356069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009420 con 0x7f00dc09d690 2024-09-13T17:43:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00db7fe640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00cb40 tx=0x7f00cc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00cc00ab30 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc00ac90 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc0a2c20 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f00cc012400 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc1373b0 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc0a3f70 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f00cc012560 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 0x7f00c8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f00cc0474b0 con 0x7f00dc09d690 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00daffd640 1 -- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 msgr2=0x7f00c8037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:20.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.394+0000 7f00daffd640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 0x7f00c8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:20.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.402+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f00dc0a3f70 con 0x7f00dc09d690 2024-09-13T17:43:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00dc138050 con 0x7f00dc09d690 2024-09-13T17:43:20.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00c3fff640 1 -- 172.21.15.96:0/3800356069 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00dc138050 con 0x7f00dc09d690 2024-09-13T17:43:20.555 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:20.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 msgr2=0x7f00c8037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:20.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00e210c640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 0x7f00c8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 msgr2=0x7f00dc1c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.550+0000 7f00e210c640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00cb40 tx=0x7f00cc00cf40 comp rx=0 tx=0).stop 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 shutdown_connections 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f00c8035570 0x7f00c8037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 --2- 172.21.15.96:0/3800356069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00dc09d690 0x7f00dc1c9820 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 >> 172.21.15.96:0/3800356069 conn(0x7f00dc06bc60 msgr2=0x7f00dc131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 shutdown_connections 2024-09-13T17:43:20.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.554+0000 7f00e210c640 1 -- 172.21.15.96:0/3800356069 wait complete. 2024-09-13T17:43:20.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:20.635 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.630+0000 7fecf5e02180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:20.688 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.682+0000 7fecf5e02180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.694+0000 7f443aac8640 1 Processor -- start 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.694+0000 7f443aac8640 1 -- start start 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f443aac8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f443409b0a0 0x7f44341350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f443aac8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4434135a30 con 0x7f44341355e0 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f443aac8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4434134af0 con 0x7f443409b0a0 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4433fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f44341355e0 0x7f4434131540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46974/0 (socket says 172.21.15.96:46974) 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4438864640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f443409b0a0 0x7f44341350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4433fff640 1 -- 172.21.15.96:0/3562467467 learned_addr learned my addr 172.21.15.96:0/3562467467 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:20.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4438864640 1 -- 172.21.15.96:0/3562467467 >> v1:172.21.15.96:6789/0 conn(0x7f44341355e0 legacy=0x7f4434131540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f44337fe640 1 -- 172.21.15.96:0/3562467467 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98725067 0 0) 0x7f4434135a30 con 0x7f44341355e0 2024-09-13T17:43:20.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4438864640 1 -- 172.21.15.96:0/3562467467 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443409b480 con 0x7f443409b0a0 2024-09-13T17:43:20.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f4438864640 1 --2- 172.21.15.96:0/3562467467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341350a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4428009740 tx=0x7f442801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac969ec2f6caeef server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f44337fe640 1 -- 172.21.15.96:0/3562467467 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4434132460 con 0x7f443409b0a0 2024-09-13T17:43:20.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f44337fe640 1 -- 172.21.15.96:0/3562467467 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44280a7050 con 0x7f443409b0a0 2024-09-13T17:43:20.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.698+0000 7f44337fe640 1 -- 172.21.15.96:0/3562467467 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44280042b0 con 0x7f443409b0a0 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/3562467467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 msgr2=0x7f44341350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 --2- 172.21.15.96:0/3562467467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341350a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f4428009740 tx=0x7f442801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/3562467467 shutdown_connections 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 --2- 172.21.15.96:0/3562467467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341350a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/3562467467 >> 172.21.15.96:0/3562467467 conn(0x7f443406e020 msgr2=0x7f4434070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/3562467467 shutdown_connections 2024-09-13T17:43:20.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/3562467467 wait complete. 2024-09-13T17:43:20.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 Processor -- start 2024-09-13T17:43:20.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- start start 2024-09-13T17:43:20.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44340a0070 con 0x7f443409b0a0 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4438864640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4438864640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48066/0 (socket says 172.21.15.96:48066) 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4438864640 1 -- 172.21.15.96:0/2409849774 learned_addr learned my addr 172.21.15.96:0/2409849774 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4438864640 1 -- 172.21.15.96:0/2409849774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428009420 con 0x7f443409b0a0 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4438864640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4428019fc0 tx=0x7f4428019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:20.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4428003aa0 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4428003c00 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44280ab440 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443409e1a0 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443409fcd0 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443409ecf0 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f44280b2040 con 0x7f443409b0a0 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 0x7f4418037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:20.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f44280e1cb0 con 0x7f443409b0a0 2024-09-13T17:43:20.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4433fff640 1 -- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 msgr2=0x7f4418037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:20.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.702+0000 7f4433fff640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 0x7f4418037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:20.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.710+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f44280aa7b0 con 0x7f443409b0a0 2024-09-13T17:43:20.739 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.734+0000 7fecf5e02180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:43:20.815 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.810+0000 7fecf5e02180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:43:20.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.854+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f443409c200 con 0x7f443409b0a0 2024-09-13T17:43:20.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.854+0000 7f44317fa640 1 -- 172.21.15.96:0/2409849774 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f44280b0070 con 0x7f443409b0a0 2024-09-13T17:43:20.861 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 msgr2=0x7f4418037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 0x7f4418037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 msgr2=0x7f44341ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4428019fc0 tx=0x7f4428019130 comp rx=0 tx=0).stop 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 shutdown_connections 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f4418035570 0x7f4418037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 --2- 172.21.15.96:0/2409849774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f443409b0a0 0x7f44341ca140 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:20.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 >> 172.21.15.96:0/2409849774 conn(0x7f443406e020 msgr2=0x7f44341342f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:20.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 shutdown_connections 2024-09-13T17:43:20.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.858+0000 7f443aac8640 1 -- 172.21.15.96:0/2409849774 wait complete. 2024-09-13T17:43:20.865 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.858+0000 7fecf5e02180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:43:20.880 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:20.944 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.938+0000 7fecf5e02180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:43:20.994 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:20.990+0000 7fecf5e02180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:43:21.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.998+0000 7ff7bc67d640 1 Processor -- start 2024-09-13T17:43:21.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:20.998+0000 7ff7bc67d640 1 -- start start 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff7b409ac70 0x7ff7b409b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40a7a80 con 0x7ff7b409bd60 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b409ef00 con 0x7ff7b409ac70 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7ba419640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff7b409ac70 0x7ff7b409b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9c18640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff7b409bd60 0x7ff7b4135d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46976/0 (socket says 172.21.15.96:46976) 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9c18640 1 -- 172.21.15.96:0/1414684336 learned_addr learned my addr 172.21.15.96:0/1414684336 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:21.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7ba419640 1 -- 172.21.15.96:0/1414684336 >> v1:172.21.15.96:6789/0 conn(0x7ff7b409bd60 legacy=0x7ff7b4135d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9417640 1 -- 172.21.15.96:0/1414684336 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461493199 0 0) 0x7ff7b40a7a80 con 0x7ff7b409bd60 2024-09-13T17:43:21.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7ba419640 1 -- 172.21.15.96:0/1414684336 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b406a890 con 0x7ff7b409ac70 2024-09-13T17:43:21.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7ba419640 1 --2- 172.21.15.96:0/1414684336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b409b050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009740 tx=0x7ff7b001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cced2ce69a8c50d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9417640 1 -- 172.21.15.96:0/1414684336 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b00ab410 con 0x7ff7b409ac70 2024-09-13T17:43:21.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9417640 1 -- 172.21.15.96:0/1414684336 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b00ab570 con 0x7ff7b409ac70 2024-09-13T17:43:21.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7b9417640 1 -- 172.21.15.96:0/1414684336 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b00ab830 con 0x7ff7b409ac70 2024-09-13T17:43:21.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- 172.21.15.96:0/1414684336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 msgr2=0x7ff7b409b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/1414684336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b409b050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0009740 tx=0x7ff7b001c3f0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- 172.21.15.96:0/1414684336 shutdown_connections 2024-09-13T17:43:21.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/1414684336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b409b050 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- 172.21.15.96:0/1414684336 >> 172.21.15.96:0/1414684336 conn(0x7ff7b406e020 msgr2=0x7ff7b4070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.002+0000 7ff7bc67d640 1 -- 172.21.15.96:0/1414684336 shutdown_connections 2024-09-13T17:43:21.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- 172.21.15.96:0/1414684336 wait complete. 2024-09-13T17:43:21.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 Processor -- start 2024-09-13T17:43:21.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- start start 2024-09-13T17:43:21.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b4120620 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7ba419640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7ba419640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48084/0 (socket says 172.21.15.96:48084) 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7ba419640 1 -- 172.21.15.96:0/372517146 learned_addr learned my addr 172.21.15.96:0/372517146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7ba419640 1 -- 172.21.15.96:0/372517146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009420 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7ba419640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00ad040 tx=0x7ff7b001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b00b5610 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b00b5770 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b4120830 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7b00b5a50 con 0x7ff7b409ac70 2024-09-13T17:43:21.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b411ec80 con 0x7ff7b409ac70 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7ff7b00b5bb0 con 0x7ff7b409ac70 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 0x7ff798037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7ff7b00e2a30 con 0x7ff7b409ac70 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7b9c18640 1 -- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 msgr2=0x7ff798037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4137860 con 0x7ff7b409ac70 2024-09-13T17:43:21.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.006+0000 7ff7b9c18640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 0x7ff798037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:21.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.014+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff7b00b5e60 con 0x7ff7b409ac70 2024-09-13T17:43:21.137 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.130+0000 7fecf5e02180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:43:21.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.170+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7b409b050 con 0x7ff7b409ac70 2024-09-13T17:43:21.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7aaffd640 1 -- 172.21.15.96:0/372517146 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v26) v1 ==== 80+0+352 (secure 0 0 0) 0x7ff7b00b3390 con 0x7ff7b409ac70 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3080912124 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/762662892 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/762662892 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/4173745824 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1797507976 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2609370247 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:21.178 INFO:teuthology.orchestra.run.smithi096.stderr:listed 6 entries 2024-09-13T17:43:21.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 msgr2=0x7ff798037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:21.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 0x7ff798037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 msgr2=0x7ff7b411dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00ad040 tx=0x7ff7b001ac30 comp rx=0 tx=0).stop 2024-09-13T17:43:21.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 shutdown_connections 2024-09-13T17:43:21.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff798035570 0x7ff798037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 --2- 172.21.15.96:0/372517146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7b409ac70 0x7ff7b411dbc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 >> 172.21.15.96:0/372517146 conn(0x7ff7b406e020 msgr2=0x7ff7b409a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 shutdown_connections 2024-09-13T17:43:21.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.174+0000 7ff7bc67d640 1 -- 172.21.15.96:0/372517146 wait complete. 2024-09-13T17:43:21.189 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.182+0000 7fecf5e02180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:43:21.196 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:43:21.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:21.197 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:21.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[36148]: fuse finished with error 0 and tester_r 0 2024-09-13T17:43:21.210 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:43:21.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:21.210 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:43:21.321 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:21.321 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:21.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:21.416 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:43:21.417 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:43:21.417 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:43:21.424 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.418+0000 7fecf5e02180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:43:21.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e12d58640 1 Processor -- start 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e12d58640 1 -- start start 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e12d58640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e0c09d690 0x7f5e0c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e12d58640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c138050 con 0x7f5e0c1387d0 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e12d58640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0c09c410 con 0x7f5e0c09d690 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e10af4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e0c09d690 0x7f5e0c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e10af4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e0c09d690 0x7f5e0c138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48090/0 (socket says 172.21.15.96:48090) 2024-09-13T17:43:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e10af4640 1 -- 172.21.15.96:0/1665790578 learned_addr learned my addr 172.21.15.96:0/1665790578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e10af4640 1 -- 172.21.15.96:0/1665790578 >> v1:172.21.15.96:6789/0 conn(0x7f5e0c1387d0 legacy=0x7f5e0c134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.482+0000 7f5e10af4640 1 -- 172.21.15.96:0/1665790578 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c09cba0 con 0x7f5e0c09d690 2024-09-13T17:43:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e10af4640 1 --2- 172.21.15.96:0/1665790578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c138290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5e000097f0 tx=0x7f5e0001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9f80803ade5ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e0b7fe640 1 -- 172.21.15.96:0/1665790578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e000a8900 con 0x7f5e0c09d690 2024-09-13T17:43:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e0b7fe640 1 -- 172.21.15.96:0/1665790578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e000a8a60 con 0x7f5e0c09d690 2024-09-13T17:43:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e0b7fe640 1 -- 172.21.15.96:0/1665790578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e000a8d20 con 0x7f5e0c09d690 2024-09-13T17:43:21.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- 172.21.15.96:0/1665790578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 msgr2=0x7f5e0c138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 --2- 172.21.15.96:0/1665790578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c138290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5e000097f0 tx=0x7f5e0001ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- 172.21.15.96:0/1665790578 shutdown_connections 2024-09-13T17:43:21.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 --2- 172.21.15.96:0/1665790578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c138290 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- 172.21.15.96:0/1665790578 >> 172.21.15.96:0/1665790578 conn(0x7f5e0c06bc60 msgr2=0x7f5e0c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- 172.21.15.96:0/1665790578 shutdown_connections 2024-09-13T17:43:21.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- 172.21.15.96:0/1665790578 wait complete. 2024-09-13T17:43:21.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 Processor -- start 2024-09-13T17:43:21.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.486+0000 7f5e12d58640 1 -- start start 2024-09-13T17:43:21.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e12d58640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e12d58640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c0a5d30 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e10af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e10af4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48098/0 (socket says 172.21.15.96:48098) 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e10af4640 1 -- 172.21.15.96:0/236015698 learned_addr learned my addr 172.21.15.96:0/236015698 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e10af4640 1 -- 172.21.15.96:0/236015698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e000094d0 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e10af4640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5e000b3040 tx=0x7f5e0001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e000b0880 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e000b09e0 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c0a6040 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e000b0c80 con 0x7f5e0c09d690 2024-09-13T17:43:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c0a44e0 con 0x7f5e0c09d690 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f5e000b7040 con 0x7f5e0c09d690 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0c09bc00 con 0x7f5e0c09d690 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 0x7f5df8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f5e000e5590 con 0x7f5e0c09d690 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e0bfff640 1 -- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 msgr2=0x7f5df8037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:21.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.490+0000 7f5e0bfff640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 0x7f5df8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:21.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.494+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e0c09bc00 con 0x7f5e0c09d690 2024-09-13T17:43:21.561 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.554+0000 7fecf5e02180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:21.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.642+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e0c09bc00 con 0x7f5e0c09d690 2024-09-13T17:43:21.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e097fa640 1 -- 172.21.15.96:0/236015698 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+10866 (secure 0 0 0) 0x7f5e0c09bc00 con 0x7f5e0c09d690 2024-09-13T17:43:21.650 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:21.650 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:10.419856+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:21.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 msgr2=0x7f5df8037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 0x7f5df8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 msgr2=0x7f5e0c0a6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f5e000b3040 tx=0x7f5e0001ba70 comp rx=0 tx=0).stop 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 shutdown_connections 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f5df8035570 0x7f5df8037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 --2- 172.21.15.96:0/236015698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e0c09d690 0x7f5e0c0a6a80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 >> 172.21.15.96:0/236015698 conn(0x7f5e0c06bc60 msgr2=0x7f5e0c0994d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 shutdown_connections 2024-09-13T17:43:21.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.646+0000 7f5e12d58640 1 -- 172.21.15.96:0/236015698 wait complete. 2024-09-13T17:43:21.668 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:49.224212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:50.030926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.0800000429153442, 'score_stable': 1.0800000429153442, 'optimal_score': 1, 'raw_score_acting': 1.0800000429153442, 'raw_score_stable': 1.0800000429153442, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:21.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:21.673 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.666+0000 7fecf5e02180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:21.736 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.730+0000 7fecf5e02180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:43:21.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff497df3640 1 Processor -- start 2024-09-13T17:43:21.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff497df3640 1 -- start start 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff497df3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff49009c930 0x7ff490098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff497df3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49009aee0 con 0x7ff49009cd10 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff497df3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49009be70 con 0x7ff49009c930 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.782+0000 7ff495b8f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff49009c930 0x7ff490098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff495b8f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff49009c930 0x7ff490098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48112/0 (socket says 172.21.15.96:48112) 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff495b8f640 1 -- 172.21.15.96:0/2277532441 learned_addr learned my addr 172.21.15.96:0/2277532441 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff495b8f640 1 -- 172.21.15.96:0/2277532441 >> v1:172.21.15.96:6789/0 conn(0x7ff49009cd10 legacy=0x7ff490098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff495b8f640 1 -- 172.21.15.96:0/2277532441 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49006ae90 con 0x7ff49009c930 2024-09-13T17:43:21.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff495b8f640 1 --2- 172.21.15.96:0/2277532441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490098a60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097f0 tx=0x7ff48c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de602cd17ebf9da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff494b8d640 1 -- 172.21.15.96:0/2277532441 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0ab530 con 0x7ff49009c930 2024-09-13T17:43:21.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff494b8d640 1 -- 172.21.15.96:0/2277532441 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c0a7050 con 0x7ff49009c930 2024-09-13T17:43:21.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff494b8d640 1 -- 172.21.15.96:0/2277532441 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0ab7e0 con 0x7ff49009c930 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 -- 172.21.15.96:0/2277532441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 msgr2=0x7ff490098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 --2- 172.21.15.96:0/2277532441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490098a60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0097f0 tx=0x7ff48c01c450 comp rx=0 tx=0).stop 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 -- 172.21.15.96:0/2277532441 shutdown_connections 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 --2- 172.21.15.96:0/2277532441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490098a60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 -- 172.21.15.96:0/2277532441 >> 172.21.15.96:0/2277532441 conn(0x7ff49006e350 msgr2=0x7ff490070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 -- 172.21.15.96:0/2277532441 shutdown_connections 2024-09-13T17:43:21.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.786+0000 7ff497df3640 1 -- 172.21.15.96:0/2277532441 wait complete. 2024-09-13T17:43:21.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 Processor -- start 2024-09-13T17:43:21.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 -- start start 2024-09-13T17:43:21.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff490123010 con 0x7ff49009c930 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff495b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff495b8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48118/0 (socket says 172.21.15.96:48118) 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff495b8f640 1 -- 172.21.15.96:0/768398801 learned_addr learned my addr 172.21.15.96:0/768398801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff495b8f640 1 -- 172.21.15.96:0/768398801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0094d0 con 0x7ff49009c930 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff495b8f640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff48c01a080 tx=0x7ff48c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0b6630 con 0x7ff49009c930 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff48c0b6790 con 0x7ff49009c930 2024-09-13T17:43:21.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff48c0b6a70 con 0x7ff49009c930 2024-09-13T17:43:21.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff490071a60 con 0x7ff49009c930 2024-09-13T17:43:21.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49011fd00 con 0x7ff49009c930 2024-09-13T17:43:21.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7ff48c0b2040 con 0x7ff49009c930 2024-09-13T17:43:21.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff490123150 con 0x7ff49009c930 2024-09-13T17:43:21.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 0x7ff474037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:21.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7ff48c0e2250 con 0x7ff49009c930 2024-09-13T17:43:21.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff49538e640 1 -- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 msgr2=0x7ff474037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:21.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.790+0000 7ff49538e640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 0x7ff474037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:21.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.794+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff48c0c7940 con 0x7ff49009c930 2024-09-13T17:43:21.873 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:21.866+0000 7fecf5e02180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:21.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.954+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff4901217c0 con 0x7ff49009c930 2024-09-13T17:43:21.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.954+0000 7ff4867fc640 1 -- 172.21.15.96:0/768398801 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff48c0b0020 con 0x7ff49009c930 2024-09-13T17:43:21.959 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:21.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 msgr2=0x7ff474037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:21.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 0x7ff474037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 msgr2=0x7ff490123a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:21.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff48c01a080 tx=0x7ff48c01ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 shutdown_connections 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7ff474035570 0x7ff474037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 --2- 172.21.15.96:0/768398801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff49009c930 0x7ff490123a70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 >> 172.21.15.96:0/768398801 conn(0x7ff49006e350 msgr2=0x7ff4900a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 shutdown_connections 2024-09-13T17:43:21.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:21.958+0000 7ff497df3640 1 -- 172.21.15.96:0/768398801 wait complete. 2024-09-13T17:43:21.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:22.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b36e63640 1 Processor -- start 2024-09-13T17:43:22.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b36e63640 1 -- start start 2024-09-13T17:43:22.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b36e63640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b3009c180 0x7f9b3013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b36e63640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3009d040 con 0x7f9b30132a10 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b36e63640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b30133500 con 0x7f9b3009c180 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b3009c180 0x7f9b3013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b3009c180 0x7f9b3013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48130/0 (socket says 172.21.15.96:48130) 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b34bff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9b30132a10 0x7f9b30132df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:47006/0 (socket says 172.21.15.96:47006) 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 -- 172.21.15.96:0/3794813012 learned_addr learned my addr 172.21.15.96:0/3794813012 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 -- 172.21.15.96:0/3794813012 >> v1:172.21.15.96:6789/0 conn(0x7f9b30132a10 legacy=0x7f9b30132df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 -- 172.21.15.96:0/3794813012 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3009f080 con 0x7f9b3009c180 2024-09-13T17:43:22.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2ffff640 1 --2- 172.21.15.96:0/3794813012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b3013ae50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097f0 tx=0x7f9b1c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94a69dba3b0f8d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2f7fe640 1 -- 172.21.15.96:0/3794813012 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c0ab530 con 0x7f9b3009c180 2024-09-13T17:43:22.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2f7fe640 1 -- 172.21.15.96:0/3794813012 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b1c0a7050 con 0x7f9b3009c180 2024-09-13T17:43:22.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.098+0000 7f9b2f7fe640 1 -- 172.21.15.96:0/3794813012 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b1c0ab7e0 con 0x7f9b3009c180 2024-09-13T17:43:22.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/3794813012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 msgr2=0x7f9b3013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 --2- 172.21.15.96:0/3794813012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b3013ae50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0097f0 tx=0x7f9b1c01c450 comp rx=0 tx=0).stop 2024-09-13T17:43:22.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/3794813012 shutdown_connections 2024-09-13T17:43:22.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 --2- 172.21.15.96:0/3794813012 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b3013ae50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/3794813012 >> 172.21.15.96:0/3794813012 conn(0x7f9b3006e020 msgr2=0x7f9b30070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/3794813012 shutdown_connections 2024-09-13T17:43:22.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/3794813012 wait complete. 2024-09-13T17:43:22.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 Processor -- start 2024-09-13T17:43:22.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- start start 2024-09-13T17:43:22.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b300a1510 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b34bff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b34bff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48138/0 (socket says 172.21.15.96:48138) 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b34bff640 1 -- 172.21.15.96:0/2723191076 learned_addr learned my addr 172.21.15.96:0/2723191076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b34bff640 1 -- 172.21.15.96:0/2723191076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1c0094d0 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b34bff640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c410 tx=0x7f9b2400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b24007410 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b24007570 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b300a31c0 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b240047f0 con 0x7f9b3009c180 2024-09-13T17:43:22.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b301cbfc0 con 0x7f9b3009c180 2024-09-13T17:43:22.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f9b24004950 con 0x7f9b3009c180 2024-09-13T17:43:22.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b300a3540 con 0x7f9b3009c180 2024-09-13T17:43:22.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 0x7f9b14037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f9b240471c0 con 0x7f9b3009c180 2024-09-13T17:43:22.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2ffff640 1 -- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 msgr2=0x7f9b14037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:22.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.102+0000 7f9b2ffff640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 0x7f9b14037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:22.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.106+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9b240192c0 con 0x7f9b3009c180 2024-09-13T17:43:22.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.258+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b30073a50 con 0x7f9b3009c180 2024-09-13T17:43:22.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.258+0000 7f9b2d7fa640 1 -- 172.21.15.96:0/2723191076 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b2401ab00 con 0x7f9b3009c180 2024-09-13T17:43:22.265 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 msgr2=0x7f9b14037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 0x7f9b14037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 msgr2=0x7f9b301cc390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9b2400c410 tx=0x7f9b2400c8d0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 shutdown_connections 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9b14035570 0x7f9b14037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 --2- 172.21.15.96:0/2723191076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b3009c180 0x7f9b301cc390 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 >> 172.21.15.96:0/2723191076 conn(0x7f9b3006e020 msgr2=0x7f9b30135070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 shutdown_connections 2024-09-13T17:43:22.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.262+0000 7f9b36e63640 1 -- 172.21.15.96:0/2723191076 wait complete. 2024-09-13T17:43:22.283 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:22.365 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:22.358+0000 7fecf5e02180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:43:22.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.386+0000 7f9421202640 1 Processor -- start 2024-09-13T17:43:22.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.386+0000 7f9421202640 1 -- start start 2024-09-13T17:43:22.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9421202640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f941c0a6510 0x7f941c13af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9421202640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941c070600 con 0x7f941c0a4900 2024-09-13T17:43:22.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9421202640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941c070830 con 0x7f941c0a6510 2024-09-13T17:43:22.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f941c0a6510 0x7f941c13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f941c0a6510 0x7f941c13af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52256/0 (socket says 172.21.15.96:52256) 2024-09-13T17:43:22.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 -- 172.21.15.96:0/646807169 learned_addr learned my addr 172.21.15.96:0/646807169 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 -- 172.21.15.96:0/646807169 >> v1:172.21.15.96:6789/0 conn(0x7f941c0a4900 legacy=0x7f941c0a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 -- 172.21.15.96:0/646807169 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c0a5c70 con 0x7f941c0a6510 2024-09-13T17:43:22.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f941a59c640 1 --2- 172.21.15.96:0/646807169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a6510 0x7f941c13af50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f94100097f0 tx=0x7f941001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b703214000ee256a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9419d9b640 1 -- 172.21.15.96:0/646807169 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94100ab520 con 0x7f941c0a6510 2024-09-13T17:43:22.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9419d9b640 1 -- 172.21.15.96:0/646807169 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94100ab680 con 0x7f941c0a6510 2024-09-13T17:43:22.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.390+0000 7f9419d9b640 1 -- 172.21.15.96:0/646807169 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94100ab940 con 0x7f941c0a6510 2024-09-13T17:43:22.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/646807169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a6510 msgr2=0x7f941c13af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 --2- 172.21.15.96:0/646807169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a6510 0x7f941c13af50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f94100097f0 tx=0x7f941001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/646807169 shutdown_connections 2024-09-13T17:43:22.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 --2- 172.21.15.96:0/646807169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a6510 0x7f941c13af50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/646807169 >> 172.21.15.96:0/646807169 conn(0x7f941c06a150 msgr2=0x7f941c06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/646807169 shutdown_connections 2024-09-13T17:43:22.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/646807169 wait complete. 2024-09-13T17:43:22.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 Processor -- start 2024-09-13T17:43:22.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- start start 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941c099210 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52268/0 (socket says 172.21.15.96:52268) 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941ad9d640 1 -- 172.21.15.96:0/1662687256 learned_addr learned my addr 172.21.15.96:0/1662687256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941ad9d640 1 -- 172.21.15.96:0/1662687256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100094d0 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941ad9d640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c480 tx=0x7f940c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f940c005150 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940c0052b0 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941c133ed0 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f940c007030 con 0x7f941c0a4900 2024-09-13T17:43:22.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941c134400 con 0x7f941c0a4900 2024-09-13T17:43:22.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f940c007190 con 0x7f941c0a4900 2024-09-13T17:43:22.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941c1394f0 con 0x7f941c0a4900 2024-09-13T17:43:22.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 0x7f9408037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941a59c640 1 -- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 msgr2=0x7f9408037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:22.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f940c046c00 con 0x7f941c0a4900 2024-09-13T17:43:22.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.394+0000 7f941a59c640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 0x7f9408037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:22.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.402+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f941c1394f0 con 0x7f941c0a4900 2024-09-13T17:43:22.441 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:22.438+0000 7fecf5e02180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:43:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.558+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f941c134400 con 0x7f941c0a4900 2024-09-13T17:43:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.558+0000 7f94037fe640 1 -- 172.21.15.96:0/1662687256 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f941c134400 con 0x7f941c0a4900 2024-09-13T17:43:22.565 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 msgr2=0x7f9408037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 0x7f9408037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 msgr2=0x7f941c137710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f940c00c480 tx=0x7f940c00c940 comp rx=0 tx=0).stop 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 shutdown_connections 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f9408035570 0x7f9408037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 --2- 172.21.15.96:0/1662687256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f941c0a4900 0x7f941c137710 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 >> 172.21.15.96:0/1662687256 conn(0x7f941c06a150 msgr2=0x7f941c12f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 shutdown_connections 2024-09-13T17:43:22.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.562+0000 7f9421202640 1 -- 172.21.15.96:0/1662687256 wait complete. 2024-09-13T17:43:22.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:22.589 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:22.582+0000 7fecf5e02180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:43:22.663 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:22.658+0000 7fecf5e02180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66b5896640 1 Processor -- start 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66b5896640 1 -- start start 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66b5896640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66b0136090 0x7f66b0136470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66b5896640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b00a79f0 con 0x7f66b01369b0 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66b5896640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b006a650 con 0x7f66b0136090 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66aeffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66b0136090 0x7f66b0136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66aeffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66b0136090 0x7f66b0136470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52274/0 (socket says 172.21.15.96:52274) 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66aeffd640 1 -- 172.21.15.96:0/1488198782 learned_addr learned my addr 172.21.15.96:0/1488198782 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66aeffd640 1 -- 172.21.15.96:0/1488198782 >> v1:172.21.15.96:6789/0 conn(0x7f66b01369b0 legacy=0x7f66b009ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.698+0000 7f66aeffd640 1 -- 172.21.15.96:0/1488198782 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b009c3d0 con 0x7f66b0136090 2024-09-13T17:43:22.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 --2- 172.21.15.96:0/1488198782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b0136470 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097f0 tx=0x7f66a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39b9d146c605f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66adffb640 1 -- 172.21.15.96:0/1488198782 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66a00ab530 con 0x7f66b0136090 2024-09-13T17:43:22.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66adffb640 1 -- 172.21.15.96:0/1488198782 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a00a7050 con 0x7f66b0136090 2024-09-13T17:43:22.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66adffb640 1 -- 172.21.15.96:0/1488198782 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66a00ab7e0 con 0x7f66b0136090 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/1488198782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 msgr2=0x7f66b0136470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 --2- 172.21.15.96:0/1488198782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b0136470 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f66a00097f0 tx=0x7f66a001c450 comp rx=0 tx=0).stop 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/1488198782 shutdown_connections 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 --2- 172.21.15.96:0/1488198782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b0136470 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/1488198782 >> 172.21.15.96:0/1488198782 conn(0x7f66b006e020 msgr2=0x7f66b0070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/1488198782 shutdown_connections 2024-09-13T17:43:22.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/1488198782 wait complete. 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 Processor -- start 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- start start 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b00a3fd0 con 0x7f66b0136090 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52276/0 (socket says 172.21.15.96:52276) 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 -- 172.21.15.96:0/3451677946 learned_addr learned my addr 172.21.15.96:0/3451677946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 -- 172.21.15.96:0/3451677946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00094d0 con 0x7f66b0136090 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66aeffd640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66a001a080 tx=0x7f66a001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66a00b6420 con 0x7f66b0136090 2024-09-13T17:43:22.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a00b6580 con 0x7f66b0136090 2024-09-13T17:43:22.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b00a2780 con 0x7f66b0136090 2024-09-13T17:43:22.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66a00b6840 con 0x7f66b0136090 2024-09-13T17:43:22.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.702+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b00a42e0 con 0x7f66b0136090 2024-09-13T17:43:22.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 11) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f66a00b2040 con 0x7f66b0136090 2024-09-13T17:43:22.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b00a4120 con 0x7f66b0136090 2024-09-13T17:43:22.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66a7fff640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 0x7f6690037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:22.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3697+0+0 (secure 0 0 0) 0x7f66a00e1f10 con 0x7f66b0136090 2024-09-13T17:43:22.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66ae7fc640 1 -- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 msgr2=0x7f6690037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/2319775113 2024-09-13T17:43:22.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.706+0000 7f66ae7fc640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 0x7f6690037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:22.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.710+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66a00b69a0 con 0x7f66b0136090 2024-09-13T17:43:22.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.870+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f66b0136470 con 0x7f66b0136090 2024-09-13T17:43:22.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.870+0000 7f66a7fff640 1 -- 172.21.15.96:0/3451677946 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v26) v1 ==== 80+0+352 (secure 0 0 0) 0x7f66a00b0070 con 0x7f66b0136090 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3080912124 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/762662892 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/762662892 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/4173745824 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1797507976 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2609370247 2024-09-14T17:43:10.419791+0000 2024-09-13T17:43:22.876 INFO:teuthology.orchestra.run.smithi096.stderr:listed 6 entries 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 msgr2=0x7f6690037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 0x7f6690037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 msgr2=0x7f66b01ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f66a001a080 tx=0x7f66a001ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 shutdown_connections 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:6824/2319775113,v1:172.21.15.96:6825/2319775113] conn(0x7f6690035570 0x7f6690037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 --2- 172.21.15.96:0/3451677946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0136090 0x7f66b01ca790 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 >> 172.21.15.96:0/3451677946 conn(0x7f66b006e020 msgr2=0x7f66b01351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 shutdown_connections 2024-09-13T17:43:22.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:22.874+0000 7f66b5896640 1 -- 172.21.15.96:0/3451677946 wait complete. 2024-09-13T17:43:22.893 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:43:22.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:22.893 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:22.948 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[36255]: fuse finished with error 0 and tester_r 0 2024-09-13T17:43:22.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:43:22.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:22.949 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:43:22.961 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:22.954+0000 7fecf5e02180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:43:23.013 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.006+0000 7fecf5e02180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:43:23.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec80b79640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec80b79640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec80b79640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec80b79640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec80b79640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.101 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec7e374640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.102 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec7e374640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.102 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec7e374640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.102 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec7e374640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.102 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:23.094+0000 7fec7e374640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:23.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.194+0000 7f87c9702640 1 Processor -- start 2024-09-13T17:43:23.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.194+0000 7f87c9702640 1 -- start start 2024-09-13T17:43:23.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87c4134e90 0x7f87c4071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c4070de0 con 0x7f87c4135270 2024-09-13T17:43:23.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c4071d60 con 0x7f87c4134e90 2024-09-13T17:43:23.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87c4134e90 0x7f87c4071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c27fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f87c4135270 0x7f87c406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54698/0 (socket says 172.21.15.96:54698) 2024-09-13T17:43:23.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c27fc640 1 -- 172.21.15.96:0/1279837300 learned_addr learned my addr 172.21.15.96:0/1279837300 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:23.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c2ffd640 1 -- 172.21.15.96:0/1279837300 >> v1:172.21.15.96:6789/0 conn(0x7f87c4135270 legacy=0x7f87c406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c2ffd640 1 -- 172.21.15.96:0/1279837300 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c406f110 con 0x7f87c4134e90 2024-09-13T17:43:23.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c2ffd640 1 --2- 172.21.15.96:0/1279837300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c4071530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009740 tx=0x7f87b801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f20a936760997e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c1ffb640 1 -- 172.21.15.96:0/1279837300 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87b80ab530 con 0x7f87c4134e90 2024-09-13T17:43:23.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c1ffb640 1 -- 172.21.15.96:0/1279837300 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80a7050 con 0x7f87c4134e90 2024-09-13T17:43:23.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c1ffb640 1 -- 172.21.15.96:0/1279837300 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87b80ab7e0 con 0x7f87c4134e90 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- 172.21.15.96:0/1279837300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 msgr2=0x7f87c4071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 --2- 172.21.15.96:0/1279837300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c4071530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009740 tx=0x7f87b801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- 172.21.15.96:0/1279837300 shutdown_connections 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 --2- 172.21.15.96:0/1279837300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c4071530 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- 172.21.15.96:0/1279837300 >> 172.21.15.96:0/1279837300 conn(0x7f87c4098a60 msgr2=0x7f87c409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- 172.21.15.96:0/1279837300 shutdown_connections 2024-09-13T17:43:23.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.198+0000 7f87c9702640 1 -- 172.21.15.96:0/1279837300 wait complete. 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 Processor -- start 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 -- start start 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40a41e0 con 0x7f87c4134e90 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52400/0 (socket says 172.21.15.96:52400) 2024-09-13T17:43:23.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c2ffd640 1 -- 172.21.15.96:0/288802509 learned_addr learned my addr 172.21.15.96:0/288802509 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c2ffd640 1 -- 172.21.15.96:0/288802509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8009420 con 0x7f87c4134e90 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c2ffd640 1 --2- 172.21.15.96:0/288802509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f87b8019fc0 tx=0x7f87b801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87b80b7450 con 0x7f87c4134e90 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b80b75b0 con 0x7f87c4134e90 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c40a52a0 con 0x7f87c4134e90 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87b80b7870 con 0x7f87c4134e90 2024-09-13T17:43:23.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c413d000 con 0x7f87c4134e90 2024-09-13T17:43:23.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c40a5530 con 0x7f87c4134e90 2024-09-13T17:43:23.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f87b80b79d0 con 0x7f87c4134e90 2024-09-13T17:43:23.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.202+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f87b80e1d50 con 0x7f87c4134e90 2024-09-13T17:43:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.206+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87c40a5530 con 0x7f87c4134e90 2024-09-13T17:43:23.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.398+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87c40a5530 con 0x7f87c4134e90 2024-09-13T17:43:23.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.398+0000 7f87ab7fe640 1 -- 172.21.15.96:0/288802509 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7f87c40a5530 con 0x7f87c4134e90 2024-09-13T17:43:23.404 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:23.404 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:43:23.405 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 msgr2=0x7f87c40a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 --2- 172.21.15.96:0/288802509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f87b8019fc0 tx=0x7f87b801abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 shutdown_connections 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 --2- 172.21.15.96:0/288802509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87c4134e90 0x7f87c40a1b40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 >> 172.21.15.96:0/288802509 conn(0x7f87c4098a60 msgr2=0x7f87c4133eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 shutdown_connections 2024-09-13T17:43:23.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.402+0000 7f87c9702640 1 -- 172.21.15.96:0/288802509 wait complete. 2024-09-13T17:43:23.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 Processor -- start 2024-09-13T17:43:23.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- start start 2024-09-13T17:43:23.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97ac06f6d0 0x7f97ac134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac06e130 con 0x7f97ac09dc20 2024-09-13T17:43:23.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0a6750 con 0x7f97ac06f6d0 2024-09-13T17:43:23.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97ac06f6d0 0x7f97ac134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f97ac06f6d0 0x7f97ac134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52404/0 (socket says 172.21.15.96:52404) 2024-09-13T17:43:23.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 -- 172.21.15.96:0/2257331797 learned_addr learned my addr 172.21.15.96:0/2257331797 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:23.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97aaffd640 1 --1- 172.21.15.96:0/2257331797 >> v1:172.21.15.96:6789/0 conn(0x7f97ac09dc20 0x7f97ac134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54712/0 (socket says 172.21.15.96:54712) 2024-09-13T17:43:23.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 -- 172.21.15.96:0/2257331797 >> v1:172.21.15.96:6789/0 conn(0x7f97ac09dc20 legacy=0x7f97ac134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 -- 172.21.15.96:0/2257331797 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0712e0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97ab7fe640 1 --2- 172.21.15.96:0/2257331797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac134ab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f97940097f0 tx=0x7f979401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a39220f50f430b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97aa7fc640 1 -- 172.21.15.96:0/2257331797 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97940ab400 con 0x7f97ac06f6d0 2024-09-13T17:43:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97aa7fc640 1 -- 172.21.15.96:0/2257331797 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97940a7050 con 0x7f97ac06f6d0 2024-09-13T17:43:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97aa7fc640 1 -- 172.21.15.96:0/2257331797 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97940ab6b0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- 172.21.15.96:0/2257331797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 msgr2=0x7f97ac134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 --2- 172.21.15.96:0/2257331797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac134ab0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f97940097f0 tx=0x7f979401c450 comp rx=0 tx=0).stop 2024-09-13T17:43:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- 172.21.15.96:0/2257331797 shutdown_connections 2024-09-13T17:43:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 --2- 172.21.15.96:0/2257331797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac134ab0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- 172.21.15.96:0/2257331797 >> 172.21.15.96:0/2257331797 conn(0x7f97ac098a60 msgr2=0x7f97ac09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- 172.21.15.96:0/2257331797 shutdown_connections 2024-09-13T17:43:23.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- 172.21.15.96:0/2257331797 wait complete. 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 Processor -- start 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.538+0000 7f97b1f1f640 1 -- start start 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97b1f1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97b1f1f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1210c0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52420/0 (socket says 172.21.15.96:52420) 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97ab7fe640 1 -- 172.21.15.96:0/318256388 learned_addr learned my addr 172.21.15.96:0/318256388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97ab7fe640 1 -- 172.21.15.96:0/318256388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940094d0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97ab7fe640 1 --2- 172.21.15.96:0/318256388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97940ae040 tx=0x7f979401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97940b6420 con 0x7f97ac06f6d0 2024-09-13T17:43:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97940b6580 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ac0a5770 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f97940b6820 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1212d0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac0a5d30 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f97940b2040 con 0x7f97ac06f6d0 2024-09-13T17:43:23.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.542+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f97940e19e0 con 0x7f97ac06f6d0 2024-09-13T17:43:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.546+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f97ac0a5d30 con 0x7f97ac06f6d0 2024-09-13T17:43:23.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.694+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97ac0a5d30 con 0x7f97ac06f6d0 2024-09-13T17:43:23.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.694+0000 7f97a3fff640 1 -- 172.21.15.96:0/318256388 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11366 (secure 0 0 0) 0x7f97ac0a5d30 con 0x7f97ac06f6d0 2024-09-13T17:43:23.700 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:23.700 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:23.031731+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2411656692":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3139097809":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3902849060":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/512890225":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1742653226":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/483607202":"2024-09-14T17:43:23.031697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:23.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 msgr2=0x7f97ac11cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 --2- 172.21.15.96:0/318256388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f97940ae040 tx=0x7f979401ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:23.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 shutdown_connections 2024-09-13T17:43:23.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 --2- 172.21.15.96:0/318256388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97ac06f6d0 0x7f97ac11cea0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 >> 172.21.15.96:0/318256388 conn(0x7f97ac098a60 msgr2=0x7f97ac1396e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 shutdown_connections 2024-09-13T17:43:23.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.698+0000 7f97b1f1f640 1 -- 172.21.15.96:0/318256388 wait complete. 2024-09-13T17:43:23.717 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:42:49.224212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:42:50.030926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.0800000429153442, 'score_stable': 1.0800000429153442, 'optimal_score': 1, 'raw_score_acting': 1.0800000429153442, 'raw_score_stable': 1.0800000429153442, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:23.718 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:23.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 Processor -- start 2024-09-13T17:43:23.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- start start 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f14ec13af50 0x7f14ec13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec138050 con 0x7f14ec09d690 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec09c410 con 0x7f14ec13af50 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f14ec13af50 0x7f14ec13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f14ec13af50 0x7f14ec13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52432/0 (socket says 172.21.15.96:52432) 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f184e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f14ec09d690 0x7f14ec138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54726/0 (socket says 172.21.15.96:54726) 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 -- 172.21.15.96:0/1406302761 learned_addr learned my addr 172.21.15.96:0/1406302761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 -- 172.21.15.96:0/1406302761 >> v1:172.21.15.96:6789/0 conn(0x7f14ec09d690 legacy=0x7f14ec138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 -- 172.21.15.96:0/1406302761 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec09cba0 con 0x7f14ec13af50 2024-09-13T17:43:23.830 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f084c640 1 -- 172.21.15.96:0/1406302761 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293396748 0 0) 0x7f14e40014a0 con 0x7f14ec09d690 2024-09-13T17:43:23.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f104d640 1 --2- 172.21.15.96:0/1406302761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec13af50 0x7f14ec13d340 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0097f0 tx=0x7f14dc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca6903623f40fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f084c640 1 -- 172.21.15.96:0/1406302761 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14dc0a8900 con 0x7f14ec13af50 2024-09-13T17:43:23.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f084c640 1 -- 172.21.15.96:0/1406302761 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc0a8a60 con 0x7f14ec13af50 2024-09-13T17:43:23.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f084c640 1 -- 172.21.15.96:0/1406302761 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14dc0a8d20 con 0x7f14ec13af50 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- 172.21.15.96:0/1406302761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec13af50 msgr2=0x7f14ec13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 --2- 172.21.15.96:0/1406302761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec13af50 0x7f14ec13d340 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f14dc0097f0 tx=0x7f14dc01ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- 172.21.15.96:0/1406302761 shutdown_connections 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 --2- 172.21.15.96:0/1406302761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec13af50 0x7f14ec13d340 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- 172.21.15.96:0/1406302761 >> 172.21.15.96:0/1406302761 conn(0x7f14ec06bc60 msgr2=0x7f14ec06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- 172.21.15.96:0/1406302761 shutdown_connections 2024-09-13T17:43:23.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.826+0000 7f14f3ab2640 1 -- 172.21.15.96:0/1406302761 wait complete. 2024-09-13T17:43:23.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 Processor -- start 2024-09-13T17:43:23.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 -- start start 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0a4c60 con 0x7f14ec09d690 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f184e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f184e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52444/0 (socket says 172.21.15.96:52444) 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f184e640 1 -- 172.21.15.96:0/538364061 learned_addr learned my addr 172.21.15.96:0/538364061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f184e640 1 -- 172.21.15.96:0/538364061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14dc0094d0 con 0x7f14ec09d690 2024-09-13T17:43:23.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f184e640 1 --2- 172.21.15.96:0/538364061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c700 tx=0x7f14e400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14e4004d80 con 0x7f14ec09d690 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e400a660 con 0x7f14ec09d690 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec0a5ee0 con 0x7f14ec09d690 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14e400a940 con 0x7f14ec09d690 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec1372d0 con 0x7f14ec09d690 2024-09-13T17:43:23.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f14e400aaa0 con 0x7f14ec09d690 2024-09-13T17:43:23.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec09bc00 con 0x7f14ec09d690 2024-09-13T17:43:23.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.830+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f14e4046bd0 con 0x7f14ec09d690 2024-09-13T17:43:23.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.834+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14ec09bc00 con 0x7f14ec09d690 2024-09-13T17:43:23.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.978+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14ec138050 con 0x7f14ec09d690 2024-09-13T17:43:23.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.978+0000 7f14e27fc640 1 -- 172.21.15.96:0/538364061 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14ec138050 con 0x7f14ec09d690 2024-09-13T17:43:23.985 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 msgr2=0x7f14ec0a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 --2- 172.21.15.96:0/538364061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f14e400c700 tx=0x7f14e400cbc0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 shutdown_connections 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 --2- 172.21.15.96:0/538364061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f14ec09d690 0x7f14ec0a6900 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 >> 172.21.15.96:0/538364061 conn(0x7f14ec06bc60 msgr2=0x7f14ec139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:23.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 shutdown_connections 2024-09-13T17:43:23.988 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:23.982+0000 7f14f3ab2640 1 -- 172.21.15.96:0/538364061 wait complete. 2024-09-13T17:43:24.001 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:24.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.114+0000 7fcf16260640 1 Processor -- start 2024-09-13T17:43:24.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.114+0000 7fcf16260640 1 -- start start 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcf1009dc20 0x7fcf101351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1006e130 con 0x7fcf1006f6d0 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf100a6750 con 0x7fcf1009dc20 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcf1009dc20 0x7fcf101351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcf1006f6d0 0x7fcf10134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54732/0 (socket says 172.21.15.96:54732) 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0f7fe640 1 -- 172.21.15.96:0/4082983978 learned_addr learned my addr 172.21.15.96:0/4082983978 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0effd640 1 -- 172.21.15.96:0/4082983978 >> v1:172.21.15.96:6789/0 conn(0x7fcf1006f6d0 legacy=0x7fcf10134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0effd640 1 -- 172.21.15.96:0/4082983978 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100712e0 con 0x7fcf1009dc20 2024-09-13T17:43:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0effd640 1 --2- 172.21.15.96:0/4082983978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1009dc20 0x7fcf101351c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009740 tx=0x7fcf0401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f763d22999a18f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0e7fc640 1 -- 172.21.15.96:0/4082983978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf040ab400 con 0x7fcf1009dc20 2024-09-13T17:43:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0e7fc640 1 -- 172.21.15.96:0/4082983978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf040a7050 con 0x7fcf1009dc20 2024-09-13T17:43:24.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf0e7fc640 1 -- 172.21.15.96:0/4082983978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf040ab6b0 con 0x7fcf1009dc20 2024-09-13T17:43:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- 172.21.15.96:0/4082983978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1009dc20 msgr2=0x7fcf101351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 --2- 172.21.15.96:0/4082983978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1009dc20 0x7fcf101351c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcf04009740 tx=0x7fcf0401c390 comp rx=0 tx=0).stop 2024-09-13T17:43:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- 172.21.15.96:0/4082983978 shutdown_connections 2024-09-13T17:43:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 --2- 172.21.15.96:0/4082983978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1009dc20 0x7fcf101351c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- 172.21.15.96:0/4082983978 >> 172.21.15.96:0/4082983978 conn(0x7fcf10098a60 msgr2=0x7fcf1009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- 172.21.15.96:0/4082983978 shutdown_connections 2024-09-13T17:43:24.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.118+0000 7fcf16260640 1 -- 172.21.15.96:0/4082983978 wait complete. 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 Processor -- start 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 -- start start 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf100a5700 con 0x7fcf1006f6d0 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52466/0 (socket says 172.21.15.96:52466) 2024-09-13T17:43:24.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0f7fe640 1 -- 172.21.15.96:0/1937438096 learned_addr learned my addr 172.21.15.96:0/1937438096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0f7fe640 1 -- 172.21.15.96:0/1937438096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf04009420 con 0x7fcf1006f6d0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0f7fe640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000cb20 tx=0x7fcf0000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf0000abf0 con 0x7fcf1006f6d0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0000ad50 con 0x7fcf1006f6d0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf100a36e0 con 0x7fcf1006f6d0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf00012400 con 0x7fcf1006f6d0 2024-09-13T17:43:24.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf101395d0 con 0x7fcf1006f6d0 2024-09-13T17:43:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf10139080 con 0x7fcf1006f6d0 2024-09-13T17:43:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fcf00007070 con 0x7fcf1006f6d0 2024-09-13T17:43:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 0x7fcefc037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7fcf00047560 con 0x7fcf1006f6d0 2024-09-13T17:43:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0effd640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 0x7fcefc037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.122+0000 7fcf0effd640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 0x7fcefc037760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ad000 tx=0x7fcf04008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.126+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf00007930 con 0x7fcf1006f6d0 2024-09-13T17:43:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.286+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf10134ab0 con 0x7fcf1006f6d0 2024-09-13T17:43:24.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.286+0000 7fcef7fff640 1 -- 172.21.15.96:0/1937438096 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf00017070 con 0x7fcf1006f6d0 2024-09-13T17:43:24.287 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 msgr2=0x7fcefc037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 0x7fcefc037760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcf040ad000 tx=0x7fcf04008040 comp rx=0 tx=0).stop 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 msgr2=0x7fcf100a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcf0000cb20 tx=0x7fcf0000a060 comp rx=0 tx=0).stop 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 shutdown_connections 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fcefc0352a0 0x7fcefc037760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 --2- 172.21.15.96:0/1937438096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf1006f6d0 0x7fcf100a18d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 >> 172.21.15.96:0/1937438096 conn(0x7fcf10098a60 msgr2=0x7fcf10133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 shutdown_connections 2024-09-13T17:43:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.290+0000 7fcf16260640 1 -- 172.21.15.96:0/1937438096 wait complete. 2024-09-13T17:43:24.306 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2550b8a640 1 Processor -- start 2024-09-13T17:43:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2550b8a640 1 -- start start 2024-09-13T17:43:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2550b8a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f254c09e820 0x7f254c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2550b8a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c0a68b0 con 0x7f254c134e70 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2550b8a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c09ddc0 con 0x7f254c09e820 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f254a59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f254c134e70 0x7f254c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54742/0 (socket says 172.21.15.96:54742) 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2549d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f254c09e820 0x7f254c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f254a59c640 1 -- 172.21.15.96:0/3156232936 learned_addr learned my addr 172.21.15.96:0/3156232936 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2549d9b640 1 -- 172.21.15.96:0/3156232936 >> v1:172.21.15.96:6789/0 conn(0x7f254c134e70 legacy=0x7f254c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2549d9b640 1 -- 172.21.15.96:0/3156232936 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c136970 con 0x7f254c09e820 2024-09-13T17:43:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.426+0000 7f2549d9b640 1 --2- 172.21.15.96:0/3156232936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c13ae50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2540009740 tx=0x7f254001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204d2ae9d344886d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254959a640 1 -- 172.21.15.96:0/3156232936 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25400ab530 con 0x7f254c09e820 2024-09-13T17:43:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254959a640 1 -- 172.21.15.96:0/3156232936 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25400a7050 con 0x7f254c09e820 2024-09-13T17:43:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254959a640 1 -- 172.21.15.96:0/3156232936 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25400ab7e0 con 0x7f254c09e820 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/3156232936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 msgr2=0x7f254c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 --2- 172.21.15.96:0/3156232936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c13ae50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2540009740 tx=0x7f254001c420 comp rx=0 tx=0).stop 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/3156232936 shutdown_connections 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 --2- 172.21.15.96:0/3156232936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c13ae50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/3156232936 >> 172.21.15.96:0/3156232936 conn(0x7f254c098a60 msgr2=0x7f254c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/3156232936 shutdown_connections 2024-09-13T17:43:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/3156232936 wait complete. 2024-09-13T17:43:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 Processor -- start 2024-09-13T17:43:24.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- start start 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0a3780 con 0x7f254c09e820 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52484/0 (socket says 172.21.15.96:52484) 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254a59c640 1 -- 172.21.15.96:0/2885612847 learned_addr learned my addr 172.21.15.96:0/2885612847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:24.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254a59c640 1 -- 172.21.15.96:0/2885612847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2540009420 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f254a59c640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c480 tx=0x7f253c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f253c005c20 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c005d80 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c13a5e0 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f253c0071a0 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.430+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0a34c0 con 0x7f254c09e820 2024-09-13T17:43:24.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f253c007300 con 0x7f254c09e820 2024-09-13T17:43:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c0a38c0 con 0x7f254c09e820 2024-09-13T17:43:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f253affd640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 0x7f252c037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f253c046b60 con 0x7f254c09e820 2024-09-13T17:43:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f2549d9b640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 0x7f252c037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.434+0000 7f2549d9b640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 0x7f252c037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25400ad000 tx=0x7f2540008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.438+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f254c0a38c0 con 0x7f254c09e820 2024-09-13T17:43:24.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.582+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f254c0a38c0 con 0x7f254c09e820 2024-09-13T17:43:24.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f253affd640 1 -- 172.21.15.96:0/2885612847 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f254c0a38c0 con 0x7f254c09e820 2024-09-13T17:43:24.585 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 msgr2=0x7f252c037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 0x7f252c037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f25400ad000 tx=0x7f2540008040 comp rx=0 tx=0).stop 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 msgr2=0x7f254c1cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f253c00c480 tx=0x7f253c00c940 comp rx=0 tx=0).stop 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 shutdown_connections 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f252c0352a0 0x7f252c037760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 --2- 172.21.15.96:0/2885612847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f254c09e820 0x7f254c1cda00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 >> 172.21.15.96:0/2885612847 conn(0x7f254c098a60 msgr2=0x7f254c134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 shutdown_connections 2024-09-13T17:43:24.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.586+0000 7f2550b8a640 1 -- 172.21.15.96:0/2885612847 wait complete. 2024-09-13T17:43:24.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:24.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.714+0000 7f034935c640 1 Processor -- start 2024-09-13T17:43:24.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.714+0000 7f034935c640 1 -- start start 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.714+0000 7f034935c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0344134e70 0x7f034409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.714+0000 7f034935c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034409ed60 con 0x7f0344135250 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.714+0000 7f034935c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f034409ddc0 con 0x7f0344134e70 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0342ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0344134e70 0x7f034409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f03427fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0344135250 0x7f0344135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54752/0 (socket says 172.21.15.96:54752) 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f03427fc640 1 -- 172.21.15.96:0/1089626353 learned_addr learned my addr 172.21.15.96:0/1089626353 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0342ffd640 1 -- 172.21.15.96:0/1089626353 >> v1:172.21.15.96:6789/0 conn(0x7f0344135250 legacy=0x7f0344135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0342ffd640 1 -- 172.21.15.96:0/1089626353 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03440a68b0 con 0x7f0344134e70 2024-09-13T17:43:24.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0342ffd640 1 --2- 172.21.15.96:0/1089626353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034409e820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0338009740 tx=0x7f033801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6059a5c3b9f6043 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0341ffb640 1 -- 172.21.15.96:0/1089626353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03380ab530 con 0x7f0344134e70 2024-09-13T17:43:24.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0341ffb640 1 -- 172.21.15.96:0/1089626353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03380a7050 con 0x7f0344134e70 2024-09-13T17:43:24.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f0341ffb640 1 -- 172.21.15.96:0/1089626353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03380ab7e0 con 0x7f0344134e70 2024-09-13T17:43:24.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 -- 172.21.15.96:0/1089626353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 msgr2=0x7f034409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 --2- 172.21.15.96:0/1089626353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034409e820 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0338009740 tx=0x7f033801c420 comp rx=0 tx=0).stop 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 -- 172.21.15.96:0/1089626353 shutdown_connections 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 --2- 172.21.15.96:0/1089626353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034409e820 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 -- 172.21.15.96:0/1089626353 >> 172.21.15.96:0/1089626353 conn(0x7f0344098a60 msgr2=0x7f034409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 -- 172.21.15.96:0/1089626353 shutdown_connections 2024-09-13T17:43:24.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 -- 172.21.15.96:0/1089626353 wait complete. 2024-09-13T17:43:24.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.718+0000 7f034935c640 1 Processor -- start 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 -- start start 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03440a1710 con 0x7f0344134e70 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f0342ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52510/0 (socket says 172.21.15.96:52510) 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f0342ffd640 1 -- 172.21.15.96:0/2093175480 learned_addr learned my addr 172.21.15.96:0/2093175480 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f0342ffd640 1 -- 172.21.15.96:0/2093175480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0338009420 con 0x7f0344134e70 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f0342ffd640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f033801a050 tx=0x7f03380191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03380abea0 con 0x7f0344134e70 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03380b4410 con 0x7f0344134e70 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03440a3550 con 0x7f0344134e70 2024-09-13T17:43:24.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f03380b46b0 con 0x7f0344134e70 2024-09-13T17:43:24.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03440a4270 con 0x7f0344134e70 2024-09-13T17:43:24.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03440a3350 con 0x7f0344134e70 2024-09-13T17:43:24.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f03380b2040 con 0x7f0344134e70 2024-09-13T17:43:24.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 0x7f0330037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f03380e1790 con 0x7f0344134e70 2024-09-13T17:43:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f03427fc640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 0x7f0330037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:24.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.722+0000 7f03427fc640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 0x7f0330037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0334001610 tx=0x7f0334007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:24.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.726+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f03380bc3b0 con 0x7f0344134e70 2024-09-13T17:43:24.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.902+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03441368b0 con 0x7f0344134e70 2024-09-13T17:43:24.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.902+0000 7f032b7fe640 1 -- 172.21.15.96:0/2093175480 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7f03380b0020 con 0x7f0344134e70 2024-09-13T17:43:24.905 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:24.905 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:43:24.906 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:43:24.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 msgr2=0x7f0330037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 0x7f0330037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0334001610 tx=0x7f0334007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 msgr2=0x7f034412fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f033801a050 tx=0x7f03380191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 shutdown_connections 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f03300352a0 0x7f0330037760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 --2- 172.21.15.96:0/2093175480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344134e70 0x7f034412fa60 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 >> 172.21.15.96:0/2093175480 conn(0x7f0344098a60 msgr2=0x7f0344133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 shutdown_connections 2024-09-13T17:43:24.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:24.906+0000 7f034935c640 1 -- 172.21.15.96:0/2093175480 wait complete. 2024-09-13T17:43:24.922 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:25.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f5651325640 1 Processor -- start 2024-09-13T17:43:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f5651325640 1 -- start start 2024-09-13T17:43:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f5651325640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f564c06f6d0 0x7f564c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f5651325640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564c06e130 con 0x7f564c09dc20 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f5651325640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f564c0a6750 con 0x7f564c06f6d0 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f564affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f564c06f6d0 0x7f564c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f564a7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f564c09dc20 0x7f564c134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54758/0 (socket says 172.21.15.96:54758) 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.038+0000 7f564a7fc640 1 -- 172.21.15.96:0/3638106206 learned_addr learned my addr 172.21.15.96:0/3638106206 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f564affd640 1 -- 172.21.15.96:0/3638106206 >> v1:172.21.15.96:6789/0 conn(0x7f564c09dc20 legacy=0x7f564c134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f564affd640 1 -- 172.21.15.96:0/3638106206 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564c0712e0 con 0x7f564c06f6d0 2024-09-13T17:43:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f564affd640 1 --2- 172.21.15.96:0/3638106206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c134ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5640009740 tx=0x7f564001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291132872bd45d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5649ffb640 1 -- 172.21.15.96:0/3638106206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56400ab400 con 0x7f564c06f6d0 2024-09-13T17:43:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5649ffb640 1 -- 172.21.15.96:0/3638106206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56400a7050 con 0x7f564c06f6d0 2024-09-13T17:43:25.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5649ffb640 1 -- 172.21.15.96:0/3638106206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56400ab6b0 con 0x7f564c06f6d0 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 -- 172.21.15.96:0/3638106206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 msgr2=0x7f564c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 --2- 172.21.15.96:0/3638106206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c134ab0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5640009740 tx=0x7f564001c390 comp rx=0 tx=0).stop 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 -- 172.21.15.96:0/3638106206 shutdown_connections 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 --2- 172.21.15.96:0/3638106206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c134ab0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 -- 172.21.15.96:0/3638106206 >> 172.21.15.96:0/3638106206 conn(0x7f564c098a60 msgr2=0x7f564c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 -- 172.21.15.96:0/3638106206 shutdown_connections 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.042+0000 7f5651325640 1 -- 172.21.15.96:0/3638106206 wait complete. 2024-09-13T17:43:25.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 Processor -- start 2024-09-13T17:43:25.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 -- start start 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564c0a6160 con 0x7f564c06f6d0 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52538/0 (socket says 172.21.15.96:52538) 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564affd640 1 -- 172.21.15.96:0/1449857855 learned_addr learned my addr 172.21.15.96:0/1449857855 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564affd640 1 -- 172.21.15.96:0/1449857855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5640009420 con 0x7f564c06f6d0 2024-09-13T17:43:25.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564affd640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5640019fc0 tx=0x7f564001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56400b7450 con 0x7f564c06f6d0 2024-09-13T17:43:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56400b75b0 con 0x7f564c06f6d0 2024-09-13T17:43:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56400b7850 con 0x7f564c06f6d0 2024-09-13T17:43:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f564c0a1e40 con 0x7f564c06f6d0 2024-09-13T17:43:25.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f564c0a30c0 con 0x7f564c06f6d0 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f56400b79b0 con 0x7f564c06f6d0 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f564c134ab0 con 0x7f564c06f6d0 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 0x7f5638037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f56400e2000 con 0x7f564c06f6d0 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564a7fc640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 0x7f5638037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.046+0000 7f564a7fc640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 0x7f5638037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f563c001610 tx=0x7f563c007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.050+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5640113db0 con 0x7f564c06f6d0 2024-09-13T17:43:25.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.214+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f564c06a1b0 con 0x7f564c06f6d0 2024-09-13T17:43:25.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f56337fe640 1 -- 172.21.15.96:0/1449857855 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11366 (secure 0 0 0) 0x7f56400bb070 con 0x7f564c06f6d0 2024-09-13T17:43:25.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:25.222 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:23.031731+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2411656692":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3139097809":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3902849060":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/512890225":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1742653226":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/483607202":"2024-09-14T17:43:23.031697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 msgr2=0x7f5638037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 0x7f5638037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f563c001610 tx=0x7f563c007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 msgr2=0x7f564c0a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5640019fc0 tx=0x7f564001abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 shutdown_connections 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f56380352a0 0x7f5638037760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 --2- 172.21.15.96:0/1449857855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c06f6d0 0x7f564c0a4270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 >> 172.21.15.96:0/1449857855 conn(0x7f564c098a60 msgr2=0x7f564c133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 shutdown_connections 2024-09-13T17:43:25.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.218+0000 7f5651325640 1 -- 172.21.15.96:0/1449857855 wait complete. 2024-09-13T17:43:25.236 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:43:25.236 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:25.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa4275a4640 1 Processor -- start 2024-09-13T17:43:25.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa4275a4640 1 -- start start 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa4275a4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa42006f6d0 0x7fa420134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa4275a4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42006e130 con 0x7fa42009dc20 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa4275a4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4200a6750 con 0x7fa42006f6d0 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa425340640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa42006f6d0 0x7fa420134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa425340640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa42006f6d0 0x7fa420134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52570/0 (socket says 172.21.15.96:52570) 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa425340640 1 -- 172.21.15.96:0/869656765 learned_addr learned my addr 172.21.15.96:0/869656765 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa425340640 1 -- 172.21.15.96:0/869656765 >> v1:172.21.15.96:6789/0 conn(0x7fa42009dc20 legacy=0x7fa420134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.354+0000 7fa425340640 1 -- 172.21.15.96:0/869656765 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200712e0 con 0x7fa42006f6d0 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 --2- 172.21.15.96:0/869656765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa420134ab0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097f0 tx=0x7fa41c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92af26c6877b2ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa417fff640 1 -- 172.21.15.96:0/869656765 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0ab400 con 0x7fa42006f6d0 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa417fff640 1 -- 172.21.15.96:0/869656765 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c0a7050 con 0x7fa42006f6d0 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa417fff640 1 -- 172.21.15.96:0/869656765 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0ab6b0 con 0x7fa42006f6d0 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/869656765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 msgr2=0x7fa420134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 --2- 172.21.15.96:0/869656765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa420134ab0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097f0 tx=0x7fa41c01c450 comp rx=0 tx=0).stop 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/869656765 shutdown_connections 2024-09-13T17:43:25.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 --2- 172.21.15.96:0/869656765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa420134ab0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/869656765 >> 172.21.15.96:0/869656765 conn(0x7fa420098a60 msgr2=0x7fa42009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/869656765 shutdown_connections 2024-09-13T17:43:25.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/869656765 wait complete. 2024-09-13T17:43:25.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 Processor -- start 2024-09-13T17:43:25.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- start start 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200a6160 con 0x7fa42006f6d0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52580/0 (socket says 172.21.15.96:52580) 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 -- 172.21.15.96:0/2439920553 learned_addr learned my addr 172.21.15.96:0/2439920553 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 -- 172.21.15.96:0/2439920553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c0094d0 con 0x7fa42006f6d0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa425340640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa41c01a080 tx=0x7fa41c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0b6420 con 0x7fa42006f6d0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c0b6580 con 0x7fa42006f6d0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4200a1e40 con 0x7fa42006f6d0 2024-09-13T17:43:25.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa41c0b6840 con 0x7fa42006f6d0 2024-09-13T17:43:25.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.358+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200a30c0 con 0x7fa42006f6d0 2024-09-13T17:43:25.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fa41c0b2040 con 0x7fa42006f6d0 2024-09-13T17:43:25.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa420134ab0 con 0x7fa42006f6d0 2024-09-13T17:43:25.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa415ffb640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 0x7fa404037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7fa41c0e1b20 con 0x7fa42006f6d0 2024-09-13T17:43:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa424b3f640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 0x7fa404037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.362+0000 7fa424b3f640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 0x7fa404037760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa410000f70 tx=0x7fa410007a40 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.366+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa41c0b69a0 con 0x7fa42006f6d0 2024-09-13T17:43:25.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.558+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa42006a1b0 con 0x7fa42006f6d0 2024-09-13T17:43:25.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa415ffb640 1 -- 172.21.15.96:0/2439920553 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fa41c0b0030 con 0x7fa42006f6d0 2024-09-13T17:43:25.562 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:25.562 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:43:25.562 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:43:25.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 msgr2=0x7fa404037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 0x7fa404037760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa410000f70 tx=0x7fa410007a40 comp rx=0 tx=0).stop 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 msgr2=0x7fa4200a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa41c01a080 tx=0x7fa41c01ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 shutdown_connections 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa4040352a0 0x7fa404037760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 --2- 172.21.15.96:0/2439920553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa42006f6d0 0x7fa4200a4270 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 >> 172.21.15.96:0/2439920553 conn(0x7fa420098a60 msgr2=0x7fa420133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 shutdown_connections 2024-09-13T17:43:25.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.562+0000 7fa4275a4640 1 -- 172.21.15.96:0/2439920553 wait complete. 2024-09-13T17:43:25.580 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:25.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1fb3e4640 1 Processor -- start 2024-09-13T17:43:25.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1fb3e4640 1 -- start start 2024-09-13T17:43:25.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1fb3e4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa1f406f6d0 0x7fa1f4134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1fb3e4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f406e130 con 0x7fa1f409dc20 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1fb3e4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f40a6750 con 0x7fa1f406f6d0 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1f9180640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa1f406f6d0 0x7fa1f4134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1f897f640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa1f409dc20 0x7fa1f4134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54786/0 (socket says 172.21.15.96:54786) 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1f9180640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa1f406f6d0 0x7fa1f4134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52586/0 (socket says 172.21.15.96:52586) 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.682+0000 7fa1f897f640 1 -- 172.21.15.96:0/2543750153 learned_addr learned my addr 172.21.15.96:0/2543750153 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1f9180640 1 -- 172.21.15.96:0/2543750153 >> v1:172.21.15.96:6789/0 conn(0x7fa1f409dc20 legacy=0x7fa1f4134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1f9180640 1 -- 172.21.15.96:0/2543750153 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f40712e0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1ebfff640 1 -- 172.21.15.96:0/2543750153 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561969809 0 0) 0x7fa1f406e130 con 0x7fa1f409dc20 2024-09-13T17:43:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1f9180640 1 --2- 172.21.15.96:0/2543750153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f4134ab0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009740 tx=0x7fa1f001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1af1f8179c12362 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1ebfff640 1 -- 172.21.15.96:0/2543750153 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00ab400 con 0x7fa1f406f6d0 2024-09-13T17:43:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1ebfff640 1 -- 172.21.15.96:0/2543750153 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f00a7050 con 0x7fa1f406f6d0 2024-09-13T17:43:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1ebfff640 1 -- 172.21.15.96:0/2543750153 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00ab6b0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/2543750153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 msgr2=0x7fa1f4134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/2543750153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f4134ab0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0009740 tx=0x7fa1f001c390 comp rx=0 tx=0).stop 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/2543750153 shutdown_connections 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/2543750153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f4134ab0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/2543750153 >> 172.21.15.96:0/2543750153 conn(0x7fa1f4098a60 msgr2=0x7fa1f409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/2543750153 shutdown_connections 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.686+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/2543750153 wait complete. 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 Processor -- start 2024-09-13T17:43:25.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 -- start start 2024-09-13T17:43:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40a2dd0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f9180640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f9180640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52598/0 (socket says 172.21.15.96:52598) 2024-09-13T17:43:25.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f9180640 1 -- 172.21.15.96:0/1400143904 learned_addr learned my addr 172.21.15.96:0/1400143904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f9180640 1 -- 172.21.15.96:0/1400143904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f0009420 con 0x7fa1f406f6d0 2024-09-13T17:43:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f9180640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0019fc0 tx=0x7fa1f001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00abea0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0004020 con 0x7fa1f406f6d0 2024-09-13T17:43:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa1f00042c0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f40a2ac0 con 0x7fa1f406f6d0 2024-09-13T17:43:25.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f40a3090 con 0x7fa1f406f6d0 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fa1f00b2050 con 0x7fa1f406f6d0 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4134a80 con 0x7fa1f406f6d0 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 0x7fa1d8037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7fa1f00e1a90 con 0x7fa1f406f6d0 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f897f640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 0x7fa1d8037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:25.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.690+0000 7fa1f897f640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 0x7fa1d8037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007fd0 tx=0x7fa1e4007920 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:25.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.694+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa1f00b2300 con 0x7fa1f406f6d0 2024-09-13T17:43:25.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.882+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1f406f220 con 0x7fa1f406f6d0 2024-09-13T17:43:25.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1e9ffb640 1 -- 172.21.15.96:0/1400143904 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3213 (secure 0 0 0) 0x7fa1f00b0030 con 0x7fa1f406f6d0 2024-09-13T17:43:25.886 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:25.886 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:42:51.058239+0000","modified":"2024-09-13T17:42:54.067162+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4488},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4488":{"gid":4488,"name":"b","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3530148857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3530148857},{"type":"v1","addr":"172.21.15.96:6829","nonce":3530148857}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-09-13T17:43:25.886 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 14 2024-09-13T17:43:25.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 msgr2=0x7fa1d8037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 0x7fa1d8037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007fd0 tx=0x7fa1e4007920 comp rx=0 tx=0).stop 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 msgr2=0x7fa1f40a3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0019fc0 tx=0x7fa1f001abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 shutdown_connections 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa1d80352a0 0x7fa1d8037760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 --2- 172.21.15.96:0/1400143904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1f406f6d0 0x7fa1f40a3520 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 >> 172.21.15.96:0/1400143904 conn(0x7fa1f4098a60 msgr2=0x7fa1f40993f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 shutdown_connections 2024-09-13T17:43:25.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:25.886+0000 7fa1fb3e4640 1 -- 172.21.15.96:0/1400143904 wait complete. 2024-09-13T17:43:25.902 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:26.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0966ce4640 1 Processor -- start 2024-09-13T17:43:26.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0966ce4640 1 -- start start 2024-09-13T17:43:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0966ce4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0960134e70 0x7f096009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0966ce4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096009ed60 con 0x7f0960135250 2024-09-13T17:43:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0966ce4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096009ddc0 con 0x7f0960134e70 2024-09-13T17:43:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f0964a80640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0960134e70 0x7f096009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f095ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0960135250 0x7f0960135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54798/0 (socket says 172.21.15.96:54798) 2024-09-13T17:43:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.014+0000 7f095ffff640 1 -- 172.21.15.96:0/3081306742 learned_addr learned my addr 172.21.15.96:0/3081306742 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0964a80640 1 -- 172.21.15.96:0/3081306742 >> v1:172.21.15.96:6789/0 conn(0x7f0960135250 legacy=0x7f0960135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0964a80640 1 -- 172.21.15.96:0/3081306742 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09600a68b0 con 0x7f0960134e70 2024-09-13T17:43:26.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f095f7fe640 1 -- 172.21.15.96:0/3081306742 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2664134490 0 0) 0x7f0948001120 con 0x7f0960135250 2024-09-13T17:43:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0964a80640 1 --2- 172.21.15.96:0/3081306742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096009e820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d90 tx=0x7f095801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37261058eb5f0098 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f095f7fe640 1 -- 172.21.15.96:0/3081306742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580a57b0 con 0x7f0960134e70 2024-09-13T17:43:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f095f7fe640 1 -- 172.21.15.96:0/3081306742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09580a5910 con 0x7f0960134e70 2024-09-13T17:43:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f095f7fe640 1 -- 172.21.15.96:0/3081306742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580a5bd0 con 0x7f0960134e70 2024-09-13T17:43:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 -- 172.21.15.96:0/3081306742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 msgr2=0x7f096009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 --2- 172.21.15.96:0/3081306742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096009e820 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f0958009d90 tx=0x7f095801c970 comp rx=0 tx=0).stop 2024-09-13T17:43:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 -- 172.21.15.96:0/3081306742 shutdown_connections 2024-09-13T17:43:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 --2- 172.21.15.96:0/3081306742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096009e820 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 -- 172.21.15.96:0/3081306742 >> 172.21.15.96:0/3081306742 conn(0x7f0960098a60 msgr2=0x7f096009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 -- 172.21.15.96:0/3081306742 shutdown_connections 2024-09-13T17:43:26.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 -- 172.21.15.96:0/3081306742 wait complete. 2024-09-13T17:43:26.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.018+0000 7f0966ce4640 1 Processor -- start 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 -- start start 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0960070770 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0964a80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0964a80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52618/0 (socket says 172.21.15.96:52618) 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0964a80640 1 -- 172.21.15.96:0/1080987991 learned_addr learned my addr 172.21.15.96:0/1080987991 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0964a80640 1 -- 172.21.15.96:0/1080987991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958009a70 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0964a80640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f09580b0040 tx=0x7f0958019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580b8690 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09580b87f0 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09600a44a0 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580b8a90 con 0x7f0960134e70 2024-09-13T17:43:26.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09600a55c0 con 0x7f0960134e70 2024-09-13T17:43:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f09580b4040 con 0x7f0960134e70 2024-09-13T17:43:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09601368b0 con 0x7f0960134e70 2024-09-13T17:43:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 0x7f0940037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7f09580e2800 con 0x7f0960134e70 2024-09-13T17:43:26.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095ffff640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 0x7f0940037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.022+0000 7f095ffff640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 0x7f0940037760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c10 tx=0x7f0948001320 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.026+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09580caba0 con 0x7f0960134e70 2024-09-13T17:43:26.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.038+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f09580b8bf0 con 0x7f0960134e70 2024-09-13T17:43:26.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.174+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09600713a0 con 0x7f0960134e70 2024-09-13T17:43:26.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f095d7fa640 1 -- 172.21.15.96:0/1080987991 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+11366 (secure 0 0 0) 0x7f09580b2030 con 0x7f0960134e70 2024-09-13T17:43:26.179 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:26.179 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:23.031731+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:42:49.224212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-13T17:42:50.030926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.0800000429153442,"score_stable":1.0800000429153442,"optimal_score":1,"raw_score_acting":1.0800000429153442,"raw_score_stable":1.0800000429153442,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2411656692":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3139097809":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3902849060":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/512890225":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1742653226":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/483607202":"2024-09-14T17:43:23.031697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 msgr2=0x7f0940037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 0x7f0940037760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0948000c10 tx=0x7f0948001320 comp rx=0 tx=0).stop 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 msgr2=0x7f096006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f09580b0040 tx=0x7f0958019710 comp rx=0 tx=0).stop 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 shutdown_connections 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f09400352a0 0x7f0940037760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 --2- 172.21.15.96:0/1080987991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0960134e70 0x7f096006f6d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 >> 172.21.15.96:0/1080987991 conn(0x7f0960098a60 msgr2=0x7f0960098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:26.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 shutdown_connections 2024-09-13T17:43:26.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.178+0000 7f0966ce4640 1 -- 172.21.15.96:0/1080987991 wait complete. 2024-09-13T17:43:26.196 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:43:26.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:43:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6edd22640 1 Processor -- start 2024-09-13T17:43:26.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6edd22640 1 -- start start 2024-09-13T17:43:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6edd22640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa6e809d690 0x7fa6e8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6edd22640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e8138050 con 0x7fa6e81387d0 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6edd22640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e809c410 con 0x7fa6e809d690 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa6e809d690 0x7fa6e8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa6e809d690 0x7fa6e8138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52632/0 (socket says 172.21.15.96:52632) 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa6e81387d0 0x7fa6e8134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54800/0 (socket says 172.21.15.96:54800) 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 -- 172.21.15.96:0/2564819433 learned_addr learned my addr 172.21.15.96:0/2564819433 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 -- 172.21.15.96:0/2564819433 >> v1:172.21.15.96:6789/0 conn(0x7fa6e81387d0 legacy=0x7fa6e8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 -- 172.21.15.96:0/2564819433 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e809cba0 con 0x7fa6e809d690 2024-09-13T17:43:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e77fe640 1 --2- 172.21.15.96:0/2564819433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e8138290 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0097f0 tx=0x7fa6dc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a987f6691467b070 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e67fc640 1 -- 172.21.15.96:0/2564819433 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6dc0a8900 con 0x7fa6e809d690 2024-09-13T17:43:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e67fc640 1 -- 172.21.15.96:0/2564819433 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc0a8a60 con 0x7fa6e809d690 2024-09-13T17:43:26.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.314+0000 7fa6e67fc640 1 -- 172.21.15.96:0/2564819433 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6dc0a8d20 con 0x7fa6e809d690 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/2564819433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 msgr2=0x7fa6e8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 --2- 172.21.15.96:0/2564819433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e8138290 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0097f0 tx=0x7fa6dc01ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/2564819433 shutdown_connections 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 --2- 172.21.15.96:0/2564819433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e8138290 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/2564819433 >> 172.21.15.96:0/2564819433 conn(0x7fa6e806bc60 msgr2=0x7fa6e806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/2564819433 shutdown_connections 2024-09-13T17:43:26.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/2564819433 wait complete. 2024-09-13T17:43:26.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 Processor -- start 2024-09-13T17:43:26.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- start start 2024-09-13T17:43:26.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e80a5d50 con 0x7fa6e809d690 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52638/0 (socket says 172.21.15.96:52638) 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e77fe640 1 -- 172.21.15.96:0/3768426177 learned_addr learned my addr 172.21.15.96:0/3768426177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e77fe640 1 -- 172.21.15.96:0/3768426177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc0094d0 con 0x7fa6e809d690 2024-09-13T17:43:26.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e77fe640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc01c900 tx=0x7fa6dc01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6dc0bf070 con 0x7fa6e809d690 2024-09-13T17:43:26.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6dc0b09b0 con 0x7fa6e809d690 2024-09-13T17:43:26.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e809f400 con 0x7fa6e809d690 2024-09-13T17:43:26.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6dc0b0c70 con 0x7fa6e809d690 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80a6060 con 0x7fa6e809d690 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fa6dc0b7070 con 0x7fa6e809d690 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e809bc00 con 0x7fa6e809d690 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 0x7fa6d40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4041+0+0 (secure 0 0 0) 0x7fa6dc0e5300 con 0x7fa6e809d690 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.318+0000 7fa6e6ffd640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 0x7fa6d40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.322+0000 7fa6e6ffd640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 0x7fa6d40379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8067370 tx=0x7fa6d8007a10 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.326+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa6e809bc00 con 0x7fa6e809d690 2024-09-13T17:43:26.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.522+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa6e80a2bd0 con 0x7fa6e809d690 2024-09-13T17:43:26.522 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:43:26.522+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:43:26.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6cffff640 1 -- 172.21.15.96:0/3768426177 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa6e80a2bd0 con 0x7fa6e809d690 2024-09-13T17:43:26.534 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 msgr2=0x7fa6d40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 0x7fa6d40379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8067370 tx=0x7fa6d8007a10 comp rx=0 tx=0).stop 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 msgr2=0x7fa6e80a6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc01c900 tx=0x7fa6dc01ba70 comp rx=0 tx=0).stop 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 shutdown_connections 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa6d40354e0 0x7fa6d40379a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 --2- 172.21.15.96:0/3768426177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6e809d690 0x7fa6e80a6aa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 >> 172.21.15.96:0/3768426177 conn(0x7fa6e806bc60 msgr2=0x7fa6e809e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 shutdown_connections 2024-09-13T17:43:26.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.534+0000 7fa6edd22640 1 -- 172.21.15.96:0/3768426177 wait complete. 2024-09-13T17:43:26.537 INFO:tasks.ceph.mds.b.smithi096.stderr: -3252> 2024-09-13T17:42:55.315+0000 7f1754c82640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:43:26.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:43:26.572 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:43:26.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f485290e640 1 Processor -- start 2024-09-13T17:43:26.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f485290e640 1 -- start start 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f485290e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f484c134e70 0x7f484c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f485290e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c09ed60 con 0x7f484c135250 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f485290e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c09ddc0 con 0x7f484c134e70 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f484c134e70 0x7f484c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f484c135250 0x7f484c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54816/0 (socket says 172.21.15.96:54816) 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484b7fe640 1 -- 172.21.15.96:0/3334627344 learned_addr learned my addr 172.21.15.96:0/3334627344 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484bfff640 1 -- 172.21.15.96:0/3334627344 >> v1:172.21.15.96:6789/0 conn(0x7f484c135250 legacy=0x7f484c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484bfff640 1 -- 172.21.15.96:0/3334627344 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0a68b0 con 0x7f484c134e70 2024-09-13T17:43:26.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484bfff640 1 --2- 172.21.15.96:0/3334627344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09e820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f483c009740 tx=0x7f483c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f39b938109d87e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484affd640 1 -- 172.21.15.96:0/3334627344 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f483c0ab530 con 0x7f484c134e70 2024-09-13T17:43:26.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484affd640 1 -- 172.21.15.96:0/3334627344 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0a7050 con 0x7f484c134e70 2024-09-13T17:43:26.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.666+0000 7f484affd640 1 -- 172.21.15.96:0/3334627344 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f483c0ab7e0 con 0x7f484c134e70 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/3334627344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 msgr2=0x7f484c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 --2- 172.21.15.96:0/3334627344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09e820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f483c009740 tx=0x7f483c01c420 comp rx=0 tx=0).stop 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/3334627344 shutdown_connections 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 --2- 172.21.15.96:0/3334627344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09e820 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/3334627344 >> 172.21.15.96:0/3334627344 conn(0x7f484c098a60 msgr2=0x7f484c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/3334627344 shutdown_connections 2024-09-13T17:43:26.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/3334627344 wait complete. 2024-09-13T17:43:26.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 Processor -- start 2024-09-13T17:43:26.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- start start 2024-09-13T17:43:26.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0a1dc0 con 0x7f484c134e70 2024-09-13T17:43:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52692/0 (socket says 172.21.15.96:52692) 2024-09-13T17:43:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484bfff640 1 -- 172.21.15.96:0/2951180790 learned_addr learned my addr 172.21.15.96:0/2951180790 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484bfff640 1 -- 172.21.15.96:0/2951180790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c009420 con 0x7f484c134e70 2024-09-13T17:43:26.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484bfff640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f483c01a050 tx=0x7f483c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f483c0abea0 con 0x7f484c134e70 2024-09-13T17:43:26.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0b4410 con 0x7f484c134e70 2024-09-13T17:43:26.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f483c0b46b0 con 0x7f484c134e70 2024-09-13T17:43:26.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c0a2a60 con 0x7f484c134e70 2024-09-13T17:43:26.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c13ba70 con 0x7f484c134e70 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484c12eed0 con 0x7f484c134e70 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 14) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f483c0b2040 con 0x7f484c134e70 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 0x7f48380379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4127+0+0 (secure 0 0 0) 0x7f483c0e1c50 con 0x7f484c134e70 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.670+0000 7f484b7fe640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 0x7f48380379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:26.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.674+0000 7f484b7fe640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 0x7f48380379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4840001610 tx=0x7f4840007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:26.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.678+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f483c0bc3b0 con 0x7f484c134e70 2024-09-13T17:43:26.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:26.862+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f484c1368b0 con 0x7f484c134e70 2024-09-13T17:43:27.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.530+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f483c0b0070 con 0x7f484c134e70 2024-09-13T17:43:27.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.534+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 msgr2=0x7f48380379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:27.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.534+0000 7f485290e640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 0x7f48380379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4840001610 tx=0x7f4840007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.534+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 msgr2=0x7f484c09f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.534+0000 7f4848ff9640 1 -- 172.21.15.96:0/2951180790 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f483c0bb360 con 0x7f484c134e70 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.534+0000 7f485290e640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f483c01a050 tx=0x7f483c0191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 shutdown_connections 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f48380354e0 0x7f48380379a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 --2- 172.21.15.96:0/2951180790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f484c134e70 0x7f484c09f280 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 >> 172.21.15.96:0/2951180790 conn(0x7f484c098a60 msgr2=0x7f484c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 shutdown_connections 2024-09-13T17:43:27.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.538+0000 7f485290e640 1 -- 172.21.15.96:0/2951180790 wait complete. 2024-09-13T17:43:27.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:43:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91dcfe0640 1 Processor -- start 2024-09-13T17:43:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91dcfe0640 1 -- start start 2024-09-13T17:43:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91dcfe0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91d8134e00 0x7f91d81351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91dcfe0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d8071cd0 con 0x7f91d806f6d0 2024-09-13T17:43:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91dcfe0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80a6730 con 0x7f91d8134e00 2024-09-13T17:43:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91d8134e00 0x7f91d81351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d5d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f91d806f6d0 0x7f91d806fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54822/0 (socket says 172.21.15.96:54822) 2024-09-13T17:43:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f91d8134e00 0x7f91d81351e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52700/0 (socket says 172.21.15.96:52700) 2024-09-13T17:43:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d5d9b640 1 -- 172.21.15.96:0/4094869460 learned_addr learned my addr 172.21.15.96:0/4094869460 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d659c640 1 -- 172.21.15.96:0/4094869460 >> v1:172.21.15.96:6789/0 conn(0x7f91d806f6d0 legacy=0x7f91d806fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.726+0000 7f91d659c640 1 -- 172.21.15.96:0/4094869460 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8071140 con 0x7f91d8134e00 2024-09-13T17:43:27.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 --2- 172.21.15.96:0/4094869460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8134e00 0x7f91d81351e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d90 tx=0x7f91d001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826c86fb3c748be server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:27.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d559a640 1 -- 172.21.15.96:0/4094869460 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d00a5650 con 0x7f91d8134e00 2024-09-13T17:43:27.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d559a640 1 -- 172.21.15.96:0/4094869460 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00a57b0 con 0x7f91d8134e00 2024-09-13T17:43:27.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d559a640 1 -- 172.21.15.96:0/4094869460 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d00a5a70 con 0x7f91d8134e00 2024-09-13T17:43:27.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- 172.21.15.96:0/4094869460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8134e00 msgr2=0x7f91d81351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/4094869460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8134e00 0x7f91d81351e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009d90 tx=0x7f91d001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- 172.21.15.96:0/4094869460 shutdown_connections 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/4094869460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d8134e00 0x7f91d81351e0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- 172.21.15.96:0/4094869460 >> 172.21.15.96:0/4094869460 conn(0x7f91d8098a60 msgr2=0x7f91d809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- 172.21.15.96:0/4094869460 shutdown_connections 2024-09-13T17:43:27.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- 172.21.15.96:0/4094869460 wait complete. 2024-09-13T17:43:27.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 Processor -- start 2024-09-13T17:43:27.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- start start 2024-09-13T17:43:27.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91dcfe0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80a5350 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52712/0 (socket says 172.21.15.96:52712) 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 -- 172.21.15.96:0/3537937304 learned_addr learned my addr 172.21.15.96:0/3537937304 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 -- 172.21.15.96:0/3537937304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d0009a70 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.730+0000 7f91d659c640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f91d00194f0 tx=0x7f91d0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d00b9580 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00b96e0 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d80a33a0 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f91d00b99a0 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d809a790 con 0x7f91d806f6d0 2024-09-13T17:43:27.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f91d00b9b00 con 0x7f91d806f6d0 2024-09-13T17:43:27.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d80a4f60 con 0x7f91d806f6d0 2024-09-13T17:43:27.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 0x7f91b8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:27.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4127+0+0 (secure 0 0 0) 0x7f91d00e2f20 con 0x7f91d806f6d0 2024-09-13T17:43:27.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91d5d9b640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 0x7f91b8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:27.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.734+0000 7f91d5d9b640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 0x7f91b8037a30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001570 tx=0x7f91cc007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:27.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.738+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f91d80a4f60 con 0x7f91d806f6d0 2024-09-13T17:43:27.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:27.898+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f91d80a4f60 con 0x7f91d806f6d0 2024-09-13T17:43:28.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.542+0000 7f91c6ffd640 1 -- 172.21.15.96:0/3537937304 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f91d80a4f60 con 0x7f91d806f6d0 2024-09-13T17:43:28.543 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:43:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 msgr2=0x7f91b8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 0x7f91b8037a30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f91cc001570 tx=0x7f91cc007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:28.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 msgr2=0x7f91d80a13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:28.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f91d00194f0 tx=0x7f91d0019680 comp rx=0 tx=0).stop 2024-09-13T17:43:28.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 shutdown_connections 2024-09-13T17:43:28.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f91b8035570 0x7f91b8037a30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:28.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 --2- 172.21.15.96:0/3537937304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f91d806f6d0 0x7f91d80a13b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:28.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 >> 172.21.15.96:0/3537937304 conn(0x7f91d8098a60 msgr2=0x7f91d8133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:28.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 shutdown_connections 2024-09-13T17:43:28.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.546+0000 7f91dcfe0640 1 -- 172.21.15.96:0/3537937304 wait complete. 2024-09-13T17:43:28.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:43:28.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.694+0000 7f25be79f640 1 Processor -- start 2024-09-13T17:43:28.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.694+0000 7f25be79f640 1 -- start start 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25be79f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f25b809e890 0x7f25b813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25be79f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b809e0d0 con 0x7f25b8135e80 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25be79f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b809ef50 con 0x7f25b809e890 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f25b809e890 0x7f25b813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b7fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f25b8135e80 0x7f25b8136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54824/0 (socket says 172.21.15.96:54824) 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b7fff640 1 -- 172.21.15.96:0/3147440388 learned_addr learned my addr 172.21.15.96:0/3147440388 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:28.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b77fe640 1 -- 172.21.15.96:0/3147440388 >> v1:172.21.15.96:6789/0 conn(0x7f25b8135e80 legacy=0x7f25b8136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:28.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b77fe640 1 -- 172.21.15.96:0/3147440388 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b809bf40 con 0x7f25b809e890 2024-09-13T17:43:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b77fe640 1 --2- 172.21.15.96:0/3147440388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b813ade0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009d90 tx=0x7f25b001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127fb1c6b0e24b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b6ffd640 1 -- 172.21.15.96:0/3147440388 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b00a58e0 con 0x7f25b809e890 2024-09-13T17:43:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b6ffd640 1 -- 172.21.15.96:0/3147440388 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b00a5a40 con 0x7f25b809e890 2024-09-13T17:43:28.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.698+0000 7f25b6ffd640 1 -- 172.21.15.96:0/3147440388 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b00a5d00 con 0x7f25b809e890 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/3147440388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 msgr2=0x7f25b813ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 --2- 172.21.15.96:0/3147440388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b813ade0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009d90 tx=0x7f25b001c970 comp rx=0 tx=0).stop 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/3147440388 shutdown_connections 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 --2- 172.21.15.96:0/3147440388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b813ade0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/3147440388 >> 172.21.15.96:0/3147440388 conn(0x7f25b8098a60 msgr2=0x7f25b809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/3147440388 shutdown_connections 2024-09-13T17:43:28.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/3147440388 wait complete. 2024-09-13T17:43:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 Processor -- start 2024-09-13T17:43:28.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- start start 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b811d770 con 0x7f25b809e890 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52736/0 (socket says 172.21.15.96:52736) 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b7fff640 1 -- 172.21.15.96:0/2094880363 learned_addr learned my addr 172.21.15.96:0/2094880363 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:28.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b7fff640 1 -- 172.21.15.96:0/2094880363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b0009a70 con 0x7f25b809e890 2024-09-13T17:43:28.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b7fff640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00c480 tx=0x7f25ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:28.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac005c20 con 0x7f25b809e890 2024-09-13T17:43:28.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac005d80 con 0x7f25b809e890 2024-09-13T17:43:28.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac0071a0 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b813a530 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.702+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b8071410 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b80008e0 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f25ac007300 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25b4ff9640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 0x7f25a4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3689+0+0 (secure 0 0 0) 0x7f25ac046e90 con 0x7f25b809e890 2024-09-13T17:43:28.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25b77fe640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 0x7f25a4037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:28.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.706+0000 7f25b77fe640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 0x7f25a4037a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25b00af000 tx=0x7f25b00a7070 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.710+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f25ac012400 con 0x7f25b809e890 2024-09-13T17:43:28.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:28.882+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f25b806f8e0 con 0x7f25b809e890 2024-09-13T17:43:29.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.546+0000 7f25b4ff9640 1 -- 172.21.15.96:0/2094880363 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f25ac017020 con 0x7f25b809e890 2024-09-13T17:43:29.550 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 msgr2=0x7f25a4037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 0x7f25a4037a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25b00af000 tx=0x7f25b00a7070 comp rx=0 tx=0).stop 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 msgr2=0x7f25b80a4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f25ac00c480 tx=0x7f25ac00c940 comp rx=0 tx=0).stop 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 shutdown_connections 2024-09-13T17:43:29.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f25a40355c0 0x7f25a4037a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 --2- 172.21.15.96:0/2094880363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25b809e890 0x7f25b80a4a70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 >> 172.21.15.96:0/2094880363 conn(0x7f25b8098a60 msgr2=0x7f25b81350a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:29.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 shutdown_connections 2024-09-13T17:43:29.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.554+0000 7f25be79f640 1 -- 172.21.15.96:0/2094880363 wait complete. 2024-09-13T17:43:29.578 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:43:29.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12f0dd5640 1 Processor -- start 2024-09-13T17:43:29.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12f0dd5640 1 -- start start 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12f0dd5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f12ec134e70 0x7f12ec09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12f0dd5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec09ed60 con 0x7f12ec135250 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12f0dd5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec09ddc0 con 0x7f12ec134e70 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12ea59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f12ec134e70 0x7f12ec09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12e9d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f12ec135250 0x7f12ec135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54830/0 (socket says 172.21.15.96:54830) 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12e9d9b640 1 -- 172.21.15.96:0/1912864950 learned_addr learned my addr 172.21.15.96:0/1912864950 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12ea59c640 1 -- 172.21.15.96:0/1912864950 >> v1:172.21.15.96:6789/0 conn(0x7f12ec135250 legacy=0x7f12ec135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.698+0000 7f12ea59c640 1 -- 172.21.15.96:0/1912864950 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0a68b0 con 0x7f12ec134e70 2024-09-13T17:43:29.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12ea59c640 1 --2- 172.21.15.96:0/1912864950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec09e820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009740 tx=0x7f12e401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e783943a40e64660 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:29.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12e959a640 1 -- 172.21.15.96:0/1912864950 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e40ab530 con 0x7f12ec134e70 2024-09-13T17:43:29.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12e959a640 1 -- 172.21.15.96:0/1912864950 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e40a7050 con 0x7f12ec134e70 2024-09-13T17:43:29.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12e959a640 1 -- 172.21.15.96:0/1912864950 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e40ab7e0 con 0x7f12ec134e70 2024-09-13T17:43:29.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 -- 172.21.15.96:0/1912864950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 msgr2=0x7f12ec09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/1912864950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec09e820 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f12e4009740 tx=0x7f12e401c420 comp rx=0 tx=0).stop 2024-09-13T17:43:29.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 -- 172.21.15.96:0/1912864950 shutdown_connections 2024-09-13T17:43:29.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/1912864950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec09e820 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 -- 172.21.15.96:0/1912864950 >> 172.21.15.96:0/1912864950 conn(0x7f12ec098a60 msgr2=0x7f12ec09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:29.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 -- 172.21.15.96:0/1912864950 shutdown_connections 2024-09-13T17:43:29.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.702+0000 7f12f0dd5640 1 -- 172.21.15.96:0/1912864950 wait complete. 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 Processor -- start 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 -- start start 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0a3690 con 0x7f12ec134e70 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52752/0 (socket says 172.21.15.96:52752) 2024-09-13T17:43:29.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12ea59c640 1 -- 172.21.15.96:0/749741265 learned_addr learned my addr 172.21.15.96:0/749741265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:29.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12ea59c640 1 -- 172.21.15.96:0/749741265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009420 con 0x7f12ec134e70 2024-09-13T17:43:29.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12ea59c640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f12e401a050 tx=0x7f12e40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e40abea0 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e40b4410 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec0a4020 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f12e40b46b0 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec0a3d60 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f12e40b2040 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec12fbf0 con 0x7f12ec134e70 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 0x7f12cc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:29.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7f12e40e1a00 con 0x7f12ec134e70 2024-09-13T17:43:29.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e9d9b640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 0x7f12cc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:29.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.706+0000 7f12e9d9b640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 0x7f12cc037a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f12d4002090 tx=0x7f12d4001680 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:29.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.710+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f12e40bc3b0 con 0x7f12ec134e70 2024-09-13T17:43:29.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.870+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f12ec1368b0 con 0x7f12ec134e70 2024-09-13T17:43:29.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12e2ffd640 1 -- 172.21.15.96:0/749741265 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v6) v1 ==== 104+0+0 (secure 0 0 0) 0x7f12e40b0070 con 0x7f12ec134e70 2024-09-13T17:43:29.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 msgr2=0x7f12cc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 0x7f12cc037a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f12d4002090 tx=0x7f12d4001680 comp rx=0 tx=0).stop 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 msgr2=0x7f12ec1cfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f12e401a050 tx=0x7f12e40191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 shutdown_connections 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f12cc035570 0x7f12cc037a30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 --2- 172.21.15.96:0/749741265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f12ec134e70 0x7f12ec1cfc50 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 >> 172.21.15.96:0/749741265 conn(0x7f12ec098a60 msgr2=0x7f12ec098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 shutdown_connections 2024-09-13T17:43:29.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:29.878+0000 7f12f0dd5640 1 -- 172.21.15.96:0/749741265 wait complete. 2024-09-13T17:43:29.897 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_fstop.TestFSTop.test_dump' 2024-09-13T17:43:30.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.014+0000 7fd7da16c640 1 Processor -- start 2024-09-13T17:43:30.017 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.014+0000 7fd7da16c640 1 -- start start 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7da16c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7d4134e70 0x7fd7d409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7da16c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d409ed60 con 0x7fd7d4135250 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7da16c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d409ddc0 con 0x7fd7d4134e70 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7d4134e70 0x7fd7d409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d2ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd7d4135250 0x7fd7d4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54836/0 (socket says 172.21.15.96:54836) 2024-09-13T17:43:30.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d2ffd640 1 -- 172.21.15.96:0/3383349889 learned_addr learned my addr 172.21.15.96:0/3383349889 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:30.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d37fe640 1 -- 172.21.15.96:0/3383349889 >> v1:172.21.15.96:6789/0 conn(0x7fd7d4135250 legacy=0x7fd7d4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d37fe640 1 -- 172.21.15.96:0/3383349889 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d40a68b0 con 0x7fd7d4134e70 2024-09-13T17:43:30.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.018+0000 7fd7d27fc640 1 -- 172.21.15.96:0/3383349889 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089155084 0 0) 0x7fd7cc0014a0 con 0x7fd7d4135250 2024-09-13T17:43:30.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 --2- 172.21.15.96:0/3383349889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d409e820 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009740 tx=0x7fd7bc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f209567abdfc9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d27fc640 1 -- 172.21.15.96:0/3383349889 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc0ab530 con 0x7fd7d4134e70 2024-09-13T17:43:30.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d27fc640 1 -- 172.21.15.96:0/3383349889 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc0a7050 con 0x7fd7d4134e70 2024-09-13T17:43:30.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d27fc640 1 -- 172.21.15.96:0/3383349889 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc0ab7e0 con 0x7fd7d4134e70 2024-09-13T17:43:30.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3383349889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 msgr2=0x7fd7d409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3383349889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d409e820 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc009740 tx=0x7fd7bc01c420 comp rx=0 tx=0).stop 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3383349889 shutdown_connections 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3383349889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d409e820 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3383349889 >> 172.21.15.96:0/3383349889 conn(0x7fd7d4098a60 msgr2=0x7fd7d409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3383349889 shutdown_connections 2024-09-13T17:43:30.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3383349889 wait complete. 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 Processor -- start 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- start start 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4070770 con 0x7fd7d4134e70 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52782/0 (socket says 172.21.15.96:52782) 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 -- 172.21.15.96:0/3671249869 learned_addr learned my addr 172.21.15.96:0/3671249869 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 -- 172.21.15.96:0/3671249869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc009420 con 0x7fd7d4134e70 2024-09-13T17:43:30.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7d37fe640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc019030 tx=0x7fd7bc01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc0abde0 con 0x7fd7d4134e70 2024-09-13T17:43:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7bc004030 con 0x7fd7d4134e70 2024-09-13T17:43:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7bc0042d0 con 0x7fd7d4134e70 2024-09-13T17:43:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d40a44a0 con 0x7fd7d4134e70 2024-09-13T17:43:30.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.022+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d40a55c0 con 0x7fd7d4134e70 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fd7bc003620 con 0x7fd7d4134e70 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d4071400 con 0x7fd7d4134e70 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7cbfff640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 0x7fd7b4037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7fd7bc0e2600 con 0x7fd7d4134e70 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7d2ffd640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 0x7fd7b4037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.026+0000 7fd7d2ffd640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 0x7fd7b4037a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000f90 tx=0x7fd7cc0016a0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.030+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7bc003e40 con 0x7fd7d4134e70 2024-09-13T17:43:30.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.246+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_dump"]} v 0) v1 -- 0x7fd7d41368b0 con 0x7fd7d4134e70 2024-09-13T17:43:30.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.550+0000 7fd7cbfff640 1 -- 172.21.15.96:0/3671249869 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_dump"]}]=0 v100) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7bc0b0070 con 0x7fd7d4134e70 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 msgr2=0x7fd7b4037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 0x7fd7b4037a30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc000f90 tx=0x7fd7cc0016a0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 msgr2=0x7fd7d406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc019030 tx=0x7fd7bc01ac60 comp rx=0 tx=0).stop 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 shutdown_connections 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fd7b4035570 0x7fd7b4037a30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 --2- 172.21.15.96:0/3671249869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d4134e70 0x7fd7d406f6d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 >> 172.21.15.96:0/3671249869 conn(0x7fd7d4098a60 msgr2=0x7fd7d4098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 shutdown_connections 2024-09-13T17:43:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.554+0000 7fd7da16c640 1 -- 172.21.15.96:0/3671249869 wait complete. 2024-09-13T17:43:30.573 INFO:tasks.cephfs_test_runner:Tests 'cephfs-top --dump' output is valid ... ok 2024-09-13T17:43:30.573 INFO:tasks.cephfs_test_runner:Starting test: test_dumpfs (tasks.cephfs.test_fstop.TestFSTop) Tests 'cephfs-top --dumpfs' output is valid 2024-09-13T17:43:30.574 INFO:tasks.cephfs_test_runner:test_dumpfs (tasks.cephfs.test_fstop.TestFSTop) 2024-09-13T17:43:30.574 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:30.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f650e536640 1 Processor -- start 2024-09-13T17:43:30.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f650e536640 1 -- start start 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f650e536640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f650809cd10 0x7f6508099170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f650e536640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650809aee0 con 0x7f650809c930 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f650e536640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650809be70 con 0x7f650809cd10 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6507fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f650809c930 0x7f6508098a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54842/0 (socket says 172.21.15.96:54842) 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6507fff640 1 -- 172.21.15.96:0/2668916503 learned_addr learned my addr 172.21.15.96:0/2668916503 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:30.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f65077fe640 1 --2- 172.21.15.96:0/2668916503 >> v2:172.21.15.96:3300/0 conn(0x7f650809cd10 0x7f6508099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6506ffd640 1 -- 172.21.15.96:0/2668916503 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1063337662 0 0) 0x7f650809aee0 con 0x7f650809c930 2024-09-13T17:43:30.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6506ffd640 1 -- 172.21.15.96:0/2668916503 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f0003530 con 0x7f650809c930 2024-09-13T17:43:30.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f65077fe640 1 -- 172.21.15.96:0/2668916503 >> v1:172.21.15.96:6789/0 conn(0x7f650809c930 legacy=0x7f6508098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f65077fe640 1 -- 172.21.15.96:0/2668916503 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650806ae90 con 0x7f650809cd10 2024-09-13T17:43:30.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f65077fe640 1 --2- 172.21.15.96:0/2668916503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809cd10 0x7f6508099170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009740 tx=0x7f64fc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930e7d0b6c2f6a36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6506ffd640 1 -- 172.21.15.96:0/2668916503 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64fc0ab530 con 0x7f650809cd10 2024-09-13T17:43:30.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6506ffd640 1 -- 172.21.15.96:0/2668916503 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc0a7050 con 0x7f650809cd10 2024-09-13T17:43:30.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.702+0000 7f6506ffd640 1 -- 172.21.15.96:0/2668916503 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64fc0ab7e0 con 0x7f650809cd10 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/2668916503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809cd10 msgr2=0x7f6508099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 --2- 172.21.15.96:0/2668916503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809cd10 0x7f6508099170 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009740 tx=0x7f64fc01c390 comp rx=0 tx=0).stop 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/2668916503 shutdown_connections 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 --2- 172.21.15.96:0/2668916503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809cd10 0x7f6508099170 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/2668916503 >> 172.21.15.96:0/2668916503 conn(0x7f650806e350 msgr2=0x7f6508070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/2668916503 shutdown_connections 2024-09-13T17:43:30.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/2668916503 wait complete. 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 Processor -- start 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- start start 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6508071820 con 0x7f650809c930 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6507fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6507fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52810/0 (socket says 172.21.15.96:52810) 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6507fff640 1 -- 172.21.15.96:0/1823853196 learned_addr learned my addr 172.21.15.96:0/1823853196 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6507fff640 1 -- 172.21.15.96:0/1823853196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc009420 con 0x7f650809c930 2024-09-13T17:43:30.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6507fff640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c450 tx=0x7f64f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f8005c20 con 0x7f650809c930 2024-09-13T17:43:30.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f8005d80 con 0x7f650809c930 2024-09-13T17:43:30.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6508123400 con 0x7f650809c930 2024-09-13T17:43:30.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f80071a0 con 0x7f650809c930 2024-09-13T17:43:30.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650811fe70 con 0x7f650809c930 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f64f8007300 con 0x7f650809c930 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6508121910 con 0x7f650809c930 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 0x7f64f4037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7f64f8046c50 con 0x7f650809c930 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.706+0000 7f65077fe640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 0x7f64f4037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.710+0000 7f65077fe640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 0x7f64f4037a30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009740 tx=0x7f64fc0a6000 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.714+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64f8012400 con 0x7f650809c930 2024-09-13T17:43:30.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.858+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6508098a60 con 0x7f650809c930 2024-09-13T17:43:30.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.858+0000 7f6504ff9640 1 -- 172.21.15.96:0/1823853196 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+7728 (secure 0 0 0) 0x7f64f8017030 con 0x7f650809c930 2024-09-13T17:43:30.860 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:30.860 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":30,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:29.538719+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6828/3530148857":"2024-09-14T17:43:26.523557+0000","172.21.15.96:0/2411656692":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1797507976":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3139097809":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/4173745824":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/2609370247":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6824/2319775113":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6824/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3902849060":"2024-09-14T17:43:23.031697+0000","172.21.15.96:6825/762662892":"2024-09-14T17:43:10.419791+0000","172.21.15.96:0/3080912124":"2024-09-14T17:43:10.419791+0000","172.21.15.96:6829/3530148857":"2024-09-14T17:43:26.523557+0000","172.21.15.96:0/512890225":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/1742653226":"2024-09-14T17:43:23.031697+0000","172.21.15.96:0/483607202":"2024-09-14T17:43:23.031697+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:30.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 msgr2=0x7f64f4037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 0x7f64f4037a30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009740 tx=0x7f64fc0a6000 comp rx=0 tx=0).stop 2024-09-13T17:43:30.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 msgr2=0x7f6508123b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f64f800c450 tx=0x7f64f800c910 comp rx=0 tx=0).stop 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 shutdown_connections 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f64f4035570 0x7f64f4037a30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 --2- 172.21.15.96:0/1823853196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650809c930 0x7f6508123b50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 >> 172.21.15.96:0/1823853196 conn(0x7f650806e350 msgr2=0x7f650813afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 shutdown_connections 2024-09-13T17:43:30.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.862+0000 7f650e536640 1 -- 172.21.15.96:0/1823853196 wait complete. 2024-09-13T17:43:30.877 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:30.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.986+0000 7fdca9ca2640 1 Processor -- start 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.986+0000 7fdca9ca2640 1 -- start start 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdca413af50 0x7fdca413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca4138050 con 0x7fdca409d690 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca409c410 con 0x7fdca413af50 2024-09-13T17:43:30.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdca413af50 0x7fdca413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdca413af50 0x7fdca413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52826/0 (socket says 172.21.15.96:52826) 2024-09-13T17:43:30.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 -- 172.21.15.96:0/2369694274 learned_addr learned my addr 172.21.15.96:0/2369694274 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:30.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 -- 172.21.15.96:0/2369694274 >> v1:172.21.15.96:6789/0 conn(0x7fdca409d690 legacy=0x7fdca4138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 -- 172.21.15.96:0/2369694274 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca409cba0 con 0x7fdca413af50 2024-09-13T17:43:30.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca27fc640 1 -- 172.21.15.96:0/2369694274 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561235433 0 0) 0x7fdc940014a0 con 0x7fdca409d690 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca2ffd640 1 --2- 172.21.15.96:0/2369694274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca413af50 0x7fdca413d340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097f0 tx=0x7fdc9801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e7e35996604174 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca27fc640 1 -- 172.21.15.96:0/2369694274 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc980a8900 con 0x7fdca413af50 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca27fc640 1 -- 172.21.15.96:0/2369694274 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc980a8a60 con 0x7fdca413af50 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca27fc640 1 -- 172.21.15.96:0/2369694274 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc980a8d20 con 0x7fdca413af50 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- 172.21.15.96:0/2369694274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca413af50 msgr2=0x7fdca413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/2369694274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca413af50 0x7fdca413d340 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdc980097f0 tx=0x7fdc9801ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- 172.21.15.96:0/2369694274 shutdown_connections 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/2369694274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca413af50 0x7fdca413d340 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- 172.21.15.96:0/2369694274 >> 172.21.15.96:0/2369694274 conn(0x7fdca406bc60 msgr2=0x7fdca406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:30.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- 172.21.15.96:0/2369694274 shutdown_connections 2024-09-13T17:43:30.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.990+0000 7fdca9ca2640 1 -- 172.21.15.96:0/2369694274 wait complete. 2024-09-13T17:43:30.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 Processor -- start 2024-09-13T17:43:30.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 -- start start 2024-09-13T17:43:30.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca40a5c90 con 0x7fdca409d690 2024-09-13T17:43:30.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52842/0 (socket says 172.21.15.96:52842) 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca37fe640 1 -- 172.21.15.96:0/161953595 learned_addr learned my addr 172.21.15.96:0/161953595 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca37fe640 1 -- 172.21.15.96:0/161953595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980094d0 con 0x7fdca409d690 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca37fe640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c6e0 tx=0x7fdc9400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc94003ab0 con 0x7fdca409d690 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94003c10 con 0x7fdca409d690 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca40a6f20 con 0x7fdca409d690 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc94005070 con 0x7fdca409d690 2024-09-13T17:43:30.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca4137400 con 0x7fdca409d690 2024-09-13T17:43:30.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdc940051d0 con 0x7fdca409d690 2024-09-13T17:43:30.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 0x7fdc90037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:30.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7fdc940473f0 con 0x7fdca409d690 2024-09-13T17:43:30.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca2ffd640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 0x7fdc90037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:30.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca4138050 con 0x7fdca409d690 2024-09-13T17:43:30.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.994+0000 7fdca2ffd640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 0x7fdc90037a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b2000 tx=0x7fdc980aa070 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:30.998+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdca4138050 con 0x7fdca409d690 2024-09-13T17:43:31.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.158+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdca4138050 con 0x7fdca409d690 2024-09-13T17:43:31.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.158+0000 7fdc8bfff640 1 -- 172.21.15.96:0/161953595 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdca4138050 con 0x7fdca409d690 2024-09-13T17:43:31.161 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 msgr2=0x7fdc90037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 0x7fdc90037a30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdc980b2000 tx=0x7fdc980aa070 comp rx=0 tx=0).stop 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 msgr2=0x7fdca40a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c6e0 tx=0x7fdc9400cba0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 shutdown_connections 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdc90035570 0x7fdc90037a30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 --2- 172.21.15.96:0/161953595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdca409d690 0x7fdca40a69e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 >> 172.21.15.96:0/161953595 conn(0x7fdca406bc60 msgr2=0x7fdca409f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:31.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 shutdown_connections 2024-09-13T17:43:31.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.162+0000 7fdca9ca2640 1 -- 172.21.15.96:0/161953595 wait complete. 2024-09-13T17:43:31.179 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs' 2024-09-13T17:43:31.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.298+0000 7efd5408d640 1 Processor -- start 2024-09-13T17:43:31.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd5408d640 1 -- start start 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd5408d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd4c09d690 0x7efd4c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd5408d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd4c138050 con 0x7efd4c1387d0 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd5408d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4c09c410 con 0x7efd4c09d690 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51e29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd4c09d690 0x7efd4c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51e29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efd4c09d690 0x7efd4c138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52846/0 (socket says 172.21.15.96:52846) 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51e29640 1 -- 172.21.15.96:0/2974936055 learned_addr learned my addr 172.21.15.96:0/2974936055 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51628640 1 --1- 172.21.15.96:0/2974936055 >> v1:172.21.15.96:6789/0 conn(0x7efd4c1387d0 0x7efd4c134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54850/0 (socket says 172.21.15.96:54850) 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51e29640 1 -- 172.21.15.96:0/2974936055 >> v1:172.21.15.96:6789/0 conn(0x7efd4c1387d0 legacy=0x7efd4c134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd51e29640 1 -- 172.21.15.96:0/2974936055 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c09cba0 con 0x7efd4c09d690 2024-09-13T17:43:31.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.302+0000 7efd50e27640 1 -- 172.21.15.96:0/2974936055 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382976405 0 0) 0x7efd3c001120 con 0x7efd4c1387d0 2024-09-13T17:43:31.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 --2- 172.21.15.96:0/2974936055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c138290 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd480097f0 tx=0x7efd4801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9735d169bfba2d4a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd50e27640 1 -- 172.21.15.96:0/2974936055 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd480a8900 con 0x7efd4c09d690 2024-09-13T17:43:31.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd50e27640 1 -- 172.21.15.96:0/2974936055 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480a8a60 con 0x7efd4c09d690 2024-09-13T17:43:31.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd50e27640 1 -- 172.21.15.96:0/2974936055 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd480a8d20 con 0x7efd4c09d690 2024-09-13T17:43:31.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/2974936055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 msgr2=0x7efd4c138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 --2- 172.21.15.96:0/2974936055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c138290 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd480097f0 tx=0x7efd4801ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/2974936055 shutdown_connections 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 --2- 172.21.15.96:0/2974936055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c138290 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/2974936055 >> 172.21.15.96:0/2974936055 conn(0x7efd4c06bc60 msgr2=0x7efd4c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/2974936055 shutdown_connections 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/2974936055 wait complete. 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 Processor -- start 2024-09-13T17:43:31.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- start start 2024-09-13T17:43:31.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c0a4360 con 0x7efd4c09d690 2024-09-13T17:43:31.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52860/0 (socket says 172.21.15.96:52860) 2024-09-13T17:43:31.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 -- 172.21.15.96:0/1748618974 learned_addr learned my addr 172.21.15.96:0/1748618974 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:31.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 -- 172.21.15.96:0/1748618974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480094d0 con 0x7efd4c09d690 2024-09-13T17:43:31.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd51e29640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd4801c900 tx=0x7efd4801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd480bf070 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd480b09b0 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efd480b0c70 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4c0a7040 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.306+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4c0a5db0 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7efd480b7070 con 0x7efd4c09d690 2024-09-13T17:43:31.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4c09bc00 con 0x7efd4c09d690 2024-09-13T17:43:31.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd427fc640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 0x7efd30037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7efd480e5040 con 0x7efd4c09d690 2024-09-13T17:43:31.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd51628640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 0x7efd30037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.310+0000 7efd51628640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 0x7efd30037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd3c000c10 tx=0x7efd3c001320 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.314+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efd4c09bc00 con 0x7efd4c09d690 2024-09-13T17:43:31.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.522+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs"]} v 0) v1 -- 0x7efd4c09bbd0 con 0x7efd4c09d690 2024-09-13T17:43:31.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.550+0000 7efd427fc640 1 -- 172.21.15.96:0/1748618974 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs"]}]=0 v101) v1 ==== 127+0+0 (secure 0 0 0) 0x7efd480b5020 con 0x7efd4c09d690 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 msgr2=0x7efd30037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 0x7efd30037a30 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efd3c000c10 tx=0x7efd3c001320 comp rx=0 tx=0).stop 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 msgr2=0x7efd4c0a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd4801c900 tx=0x7efd4801ba70 comp rx=0 tx=0).stop 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 shutdown_connections 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7efd30035570 0x7efd30037a30 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 --2- 172.21.15.96:0/1748618974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd4c09d690 0x7efd4c0a6b00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 >> 172.21.15.96:0/1748618974 conn(0x7efd4c06bc60 msgr2=0x7efd4c099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:31.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 shutdown_connections 2024-09-13T17:43:31.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.554+0000 7efd5408d640 1 -- 172.21.15.96:0/1748618974 wait complete. 2024-09-13T17:43:31.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:43:31.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7bec2640 1 Processor -- start 2024-09-13T17:43:31.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7bec2640 1 -- start start 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7bec2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1f7409e820 0x7f1f7413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7bec2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740a68b0 con 0x7f1f74134e70 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7bec2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7409ddc0 con 0x7f1f7409e820 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f7945d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1f7409e820 0x7f1f7413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f79c5e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1f74134e70 0x7f1f74135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54856/0 (socket says 172.21.15.96:54856) 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.690+0000 7f1f79c5e640 1 -- 172.21.15.96:0/594105196 learned_addr learned my addr 172.21.15.96:0/594105196 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f7945d640 1 -- 172.21.15.96:0/594105196 >> v1:172.21.15.96:6789/0 conn(0x7f1f74134e70 legacy=0x7f1f74135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f7945d640 1 -- 172.21.15.96:0/594105196 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f74136970 con 0x7f1f7409e820 2024-09-13T17:43:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f7945d640 1 --2- 172.21.15.96:0/594105196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f7413ae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009740 tx=0x7f1f6401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65490d4ea8eeb454 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f78c5c640 1 -- 172.21.15.96:0/594105196 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f640ab530 con 0x7f1f7409e820 2024-09-13T17:43:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f78c5c640 1 -- 172.21.15.96:0/594105196 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f640a7050 con 0x7f1f7409e820 2024-09-13T17:43:31.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.694+0000 7f1f78c5c640 1 -- 172.21.15.96:0/594105196 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f640ab7e0 con 0x7f1f7409e820 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/594105196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 msgr2=0x7f1f7413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/594105196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f7413ae50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009740 tx=0x7f1f6401c420 comp rx=0 tx=0).stop 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/594105196 shutdown_connections 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/594105196 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f7413ae50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/594105196 >> 172.21.15.96:0/594105196 conn(0x7f1f74098a60 msgr2=0x7f1f7409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/594105196 shutdown_connections 2024-09-13T17:43:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/594105196 wait complete. 2024-09-13T17:43:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 Processor -- start 2024-09-13T17:43:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- start start 2024-09-13T17:43:31.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740a3780 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f79c5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f79c5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52870/0 (socket says 172.21.15.96:52870) 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f79c5e640 1 -- 172.21.15.96:0/3189593360 learned_addr learned my addr 172.21.15.96:0/3189593360 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f79c5e640 1 -- 172.21.15.96:0/3189593360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f64009420 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f79c5e640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00cb20 tx=0x7f1f6c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c00abf0 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00ad50 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7413a5e0 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f6c012400 con 0x7f1f7409e820 2024-09-13T17:43:31.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740a34c0 con 0x7f1f7409e820 2024-09-13T17:43:31.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f1f6c012560 con 0x7f1f7409e820 2024-09-13T17:43:31.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.698+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f740a38c0 con 0x7f1f7409e820 2024-09-13T17:43:31.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.702+0000 7f1f6a7fc640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 0x7f1f58037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:31.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.702+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7f1f6c0476f0 con 0x7f1f7409e820 2024-09-13T17:43:31.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.702+0000 7f1f7945d640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 0x7f1f58037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:31.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.702+0000 7f1f7945d640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 0x7f1f58037a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f640ad000 tx=0x7f1f64008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:31.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.706+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f740a38c0 con 0x7f1f7409e820 2024-09-13T17:43:31.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.862+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f740a38c0 con 0x7f1f7409e820 2024-09-13T17:43:31.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.870+0000 7f1f6a7fc640 1 -- 172.21.15.96:0/3189593360 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v7)=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f740a38c0 con 0x7f1f7409e820 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.870+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 msgr2=0x7f1f58037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.870+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 0x7f1f58037a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f640ad000 tx=0x7f1f64008040 comp rx=0 tx=0).stop 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.870+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 msgr2=0x7f1f741cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00cb20 tx=0x7f1f6c00a060 comp rx=0 tx=0).stop 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 shutdown_connections 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1f58035570 0x7f1f58037a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 --2- 172.21.15.96:0/3189593360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f7409e820 0x7f1f741cda00 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 >> 172.21.15.96:0/3189593360 conn(0x7f1f74098a60 msgr2=0x7f1f74134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:31.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 shutdown_connections 2024-09-13T17:43:31.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:31.874+0000 7f1f7bec2640 1 -- 172.21.15.96:0/3189593360 wait complete. 2024-09-13T17:43:31.890 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:43:31.890 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:43:31.959 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:43:31.959 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:31.959 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:31.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:43:31.965 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:31.965 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:32.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:43:32.010 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-09-13T17:43:32.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd7bde3640 1 Processor -- start 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd7bde3640 1 -- start start 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd7bde3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd7bde3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7409ed60 con 0x7fdd74135250 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd7bde3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7409ddc0 con 0x7fdd74134e70 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd79b7f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd79b7f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52884/0 (socket says 172.21.15.96:52884) 2024-09-13T17:43:32.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd79b7f640 1 -- 172.21.15.96:0/1696243307 learned_addr learned my addr 172.21.15.96:0/1696243307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:32.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd79b7f640 1 -- 172.21.15.96:0/1696243307 >> v1:172.21.15.96:6789/0 conn(0x7fdd74135250 legacy=0x7fdd74135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.174+0000 7fdd79b7f640 1 -- 172.21.15.96:0/1696243307 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740a68b0 con 0x7fdd74134e70 2024-09-13T17:43:32.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd79b7f640 1 --2- 172.21.15.96:0/1696243307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097f0 tx=0x7fdd7001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490b004e3e575211 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd78b7d640 1 -- 172.21.15.96:0/1696243307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd700ac540 con 0x7fdd74134e70 2024-09-13T17:43:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd78b7d640 1 -- 172.21.15.96:0/1696243307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd700ac6a0 con 0x7fdd74134e70 2024-09-13T17:43:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd78b7d640 1 -- 172.21.15.96:0/1696243307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd700ac960 con 0x7fdd74134e70 2024-09-13T17:43:32.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- 172.21.15.96:0/1696243307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 msgr2=0x7fdd7409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/1696243307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd700097f0 tx=0x7fdd7001c4e0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- 172.21.15.96:0/1696243307 shutdown_connections 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/1696243307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- 172.21.15.96:0/1696243307 >> 172.21.15.96:0/1696243307 conn(0x7fdd74098a60 msgr2=0x7fdd7409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- 172.21.15.96:0/1696243307 shutdown_connections 2024-09-13T17:43:32.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- 172.21.15.96:0/1696243307 wait complete. 2024-09-13T17:43:32.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 Processor -- start 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.178+0000 7fdd7bde3640 1 -- start start 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7bde3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7bde3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740a1710 con 0x7fdd74134e70 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd79b7f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd79b7f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52892/0 (socket says 172.21.15.96:52892) 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd79b7f640 1 -- 172.21.15.96:0/4141270071 learned_addr learned my addr 172.21.15.96:0/4141270071 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd79b7f640 1 -- 172.21.15.96:0/4141270071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd700094d0 con 0x7fdd74134e70 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd79b7f640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd700190f0 tx=0x7fdd70019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd700bb070 con 0x7fdd74134e70 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd700b7470 con 0x7fdd74134e70 2024-09-13T17:43:32.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd740a3550 con 0x7fdd74134e70 2024-09-13T17:43:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd700b7730 con 0x7fdd74134e70 2024-09-13T17:43:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd740a4270 con 0x7fdd74134e70 2024-09-13T17:43:32.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdd700b3070 con 0x7fdd74134e70 2024-09-13T17:43:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd740a3350 con 0x7fdd74134e70 2024-09-13T17:43:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3275+0+0 (secure 0 0 0) 0x7fdd700e1990 con 0x7fdd74134e70 2024-09-13T17:43:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7937e640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.182+0000 7fdd7937e640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000f70 tx=0x7fdd64007a40 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.186+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd700b4380 con 0x7fdd74134e70 2024-09-13T17:43:32.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.378+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd741368b0 con 0x7fdd74134e70 2024-09-13T17:43:32.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.378+0000 7fdd6a7fc640 1 -- 172.21.15.96:0/4141270071 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2005 (secure 0 0 0) 0x7fdd700b1070 con 0x7fdd74134e70 2024-09-13T17:43:32.380 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:32.380 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/74070568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":74070568},{"type":"v1","addr":"172.21.15.96:6827","nonce":74070568}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4707,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/498335092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":498335092},{"type":"v1","addr":"172.21.15.96:6829","nonce":498335092}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[]} 2024-09-13T17:43:32.380 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 16 2024-09-13T17:43:32.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 msgr2=0x7fdd58037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000f70 tx=0x7fdd64007a40 comp rx=0 tx=0).stop 2024-09-13T17:43:32.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 msgr2=0x7fdd7412fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fdd700190f0 tx=0x7fdd70019280 comp rx=0 tx=0).stop 2024-09-13T17:43:32.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 shutdown_connections 2024-09-13T17:43:32.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 --2- 172.21.15.96:0/4141270071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7412fa60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 >> 172.21.15.96:0/4141270071 conn(0x7fdd74098a60 msgr2=0x7fdd74133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:32.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 shutdown_connections 2024-09-13T17:43:32.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.382+0000 7fdd7bde3640 1 -- 172.21.15.96:0/4141270071 wait complete. 2024-09-13T17:43:32.398 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:43:32.398 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:43:32.399 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:43:32.399 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:43:32.399 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:43:32.399 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T17:43:32.399 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:43:32.399 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:43:32.408 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:43:32.409 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:43:32.410 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:43:32.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:43:32.454 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:43:32.455 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:32.455 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:43:32.455 DEBUG:teuthology.parallel:result is None 2024-09-13T17:43:32.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:43:32.512 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:43:32.520 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:43:32.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe50a99f640 1 Processor -- start 2024-09-13T17:43:32.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe50a99f640 1 -- start start 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe50a99f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe50409dc30 0x7fe50413ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe50a99f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5041327b0 con 0x7fe5041329b0 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe50a99f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50409bd90 con 0x7fe50409dc30 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe5037fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe50409dc30 0x7fe50413ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe503fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe5041329b0 0x7fe504132d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33726/0 (socket says 172.21.15.96:33726) 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe5037fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe50409dc30 0x7fe50413ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50134/0 (socket says 172.21.15.96:50134) 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe503fff640 1 -- 172.21.15.96:0/4267167164 learned_addr learned my addr 172.21.15.96:0/4267167164 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe5037fe640 1 -- 172.21.15.96:0/4267167164 >> v1:172.21.15.96:6789/0 conn(0x7fe5041329b0 legacy=0x7fe504132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe502ffd640 1 -- 172.21.15.96:0/4267167164 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886714155 0 0) 0x7fe5041327b0 con 0x7fe5041329b0 2024-09-13T17:43:32.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.578+0000 7fe5037fe640 1 -- 172.21.15.96:0/4267167164 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50409b2a0 con 0x7fe50409dc30 2024-09-13T17:43:32.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe5037fe640 1 --2- 172.21.15.96:0/4267167164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe50413ad00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009740 tx=0x7fe4f801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37fc33923d854f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe502ffd640 1 -- 172.21.15.96:0/4267167164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f80ab530 con 0x7fe50409dc30 2024-09-13T17:43:32.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe502ffd640 1 -- 172.21.15.96:0/4267167164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f80a7050 con 0x7fe50409dc30 2024-09-13T17:43:32.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe502ffd640 1 -- 172.21.15.96:0/4267167164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f80ab7e0 con 0x7fe50409dc30 2024-09-13T17:43:32.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- 172.21.15.96:0/4267167164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 msgr2=0x7fe50413ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 --2- 172.21.15.96:0/4267167164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe50413ad00 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4f8009740 tx=0x7fe4f801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- 172.21.15.96:0/4267167164 shutdown_connections 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 --2- 172.21.15.96:0/4267167164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe50413ad00 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- 172.21.15.96:0/4267167164 >> 172.21.15.96:0/4267167164 conn(0x7fe50406ef80 msgr2=0x7fe5040713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- 172.21.15.96:0/4267167164 shutdown_connections 2024-09-13T17:43:32.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- 172.21.15.96:0/4267167164 wait complete. 2024-09-13T17:43:32.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 Processor -- start 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- start start 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe50a99f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50406f660 con 0x7fe50409dc30 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe503fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe503fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50138/0 (socket says 172.21.15.96:50138) 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe503fff640 1 -- 172.21.15.96:0/3178718954 learned_addr learned my addr 172.21.15.96:0/3178718954 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe503fff640 1 -- 172.21.15.96:0/3178718954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f8009420 con 0x7fe50409dc30 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe503fff640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c450 tx=0x7fe4f400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f4005c20 con 0x7fe50409dc30 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4f4005d80 con 0x7fe50409dc30 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.582+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4f40071a0 con 0x7fe50409dc30 2024-09-13T17:43:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5040a5430 con 0x7fe50409dc30 2024-09-13T17:43:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5040a56f0 con 0x7fe50409dc30 2024-09-13T17:43:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fe4f4007300 con 0x7fe50409dc30 2024-09-13T17:43:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5040008e0 con 0x7fe50409dc30 2024-09-13T17:43:32.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe500ff9640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 0x7fe4f0037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:32.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3275+0+0 (secure 0 0 0) 0x7fe4f4047330 con 0x7fe50409dc30 2024-09-13T17:43:32.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe5037fe640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 0x7fe4f0037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:32.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.586+0000 7fe5037fe640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 0x7fe4f0037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80ad000 tx=0x7fe4f8008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:32.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.590+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe4f40075b0 con 0x7fe50409dc30 2024-09-13T17:43:32.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:32.758+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe50409c1d0 con 0x7fe50409dc30 2024-09-13T17:43:33.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.558+0000 7fe500ff9640 1 -- 172.21.15.96:0/3178718954 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe4f4019300 con 0x7fe50409dc30 2024-09-13T17:43:33.562 INFO:teuthology.orchestra.run.smithi096.stderr: removed all blocklist entries 2024-09-13T17:43:33.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 msgr2=0x7fe4f0037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 0x7fe4f0037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe4f80ad000 tx=0x7fe4f8008040 comp rx=0 tx=0).stop 2024-09-13T17:43:33.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 msgr2=0x7fe5040a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400c450 tx=0x7fe4f400c910 comp rx=0 tx=0).stop 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 shutdown_connections 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe4f0035570 0x7fe4f0037a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 --2- 172.21.15.96:0/3178718954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe50409dc30 0x7fe5040a5e70 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 >> 172.21.15.96:0/3178718954 conn(0x7fe50406ef80 msgr2=0x7fe5040706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 shutdown_connections 2024-09-13T17:43:33.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.562+0000 7fe50a99f640 1 -- 172.21.15.96:0/3178718954 wait complete. 2024-09-13T17:43:33.585 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:43:33.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16ad380640 1 Processor -- start 2024-09-13T17:43:33.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16ad380640 1 -- start start 2024-09-13T17:43:33.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16ad380640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f16a809dc20 0x7f16a81351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16ad380640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a806e130 con 0x7f16a806f6d0 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16ad380640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a80a6750 con 0x7f16a809dc20 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f16a806f6d0 0x7f16a8134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33736/0 (socket says 172.21.15.96:33736) 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a6ffd640 1 -- 172.21.15.96:0/2366674640 learned_addr learned my addr 172.21.15.96:0/2366674640 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a67fc640 1 --2- 172.21.15.96:0/2366674640 >> v2:172.21.15.96:3300/0 conn(0x7f16a809dc20 0x7f16a81351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a5ffb640 1 -- 172.21.15.96:0/2366674640 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668357130 0 0) 0x7f16a806e130 con 0x7f16a806f6d0 2024-09-13T17:43:33.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a5ffb640 1 -- 172.21.15.96:0/2366674640 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1694005740 con 0x7f16a806f6d0 2024-09-13T17:43:33.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a67fc640 1 -- 172.21.15.96:0/2366674640 >> v1:172.21.15.96:6789/0 conn(0x7f16a806f6d0 legacy=0x7f16a8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.706+0000 7f16a67fc640 1 -- 172.21.15.96:0/2366674640 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80712e0 con 0x7f16a809dc20 2024-09-13T17:43:33.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16a67fc640 1 --2- 172.21.15.96:0/2366674640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a809dc20 0x7f16a81351c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1698009d90 tx=0x7f169801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c876a9faab6282c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16a5ffb640 1 -- 172.21.15.96:0/2366674640 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16980a5540 con 0x7f16a809dc20 2024-09-13T17:43:33.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16a5ffb640 1 -- 172.21.15.96:0/2366674640 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16980a56a0 con 0x7f16a809dc20 2024-09-13T17:43:33.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16a5ffb640 1 -- 172.21.15.96:0/2366674640 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16980a5940 con 0x7f16a809dc20 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- 172.21.15.96:0/2366674640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a809dc20 msgr2=0x7f16a81351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 --2- 172.21.15.96:0/2366674640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a809dc20 0x7f16a81351c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f1698009d90 tx=0x7f169801c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- 172.21.15.96:0/2366674640 shutdown_connections 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 --2- 172.21.15.96:0/2366674640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a809dc20 0x7f16a81351c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- 172.21.15.96:0/2366674640 >> 172.21.15.96:0/2366674640 conn(0x7f16a8098a60 msgr2=0x7f16a809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- 172.21.15.96:0/2366674640 shutdown_connections 2024-09-13T17:43:33.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- 172.21.15.96:0/2366674640 wait complete. 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 Processor -- start 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- start start 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16ad380640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a81354f0 con 0x7f16a806f6d0 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.710+0000 7f16a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50162/0 (socket says 172.21.15.96:50162) 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a6ffd640 1 -- 172.21.15.96:0/1237714516 learned_addr learned my addr 172.21.15.96:0/1237714516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a6ffd640 1 -- 172.21.15.96:0/1237714516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009a70 con 0x7f16a806f6d0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a6ffd640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f169000c3d0 tx=0x7f169000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1690005ce0 con 0x7f16a806f6d0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690005e40 con 0x7f16a806f6d0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a80a5bf0 con 0x7f16a806f6d0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1690007220 con 0x7f16a806f6d0 2024-09-13T17:43:33.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a8122330 con 0x7f16a806f6d0 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f1690007440 con 0x7f16a806f6d0 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a8121eb0 con 0x7f16a806f6d0 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 0x7f16880379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f1690046b80 con 0x7f16a806f6d0 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a67fc640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 0x7f16880379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:33.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.714+0000 7f16a67fc640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 0x7f16880379e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f16980af000 tx=0x7f16980a7070 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:33.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.718+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1690019b40 con 0x7f16a806f6d0 2024-09-13T17:43:33.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f16a81cbeb0 con 0x7f16a806f6d0 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f169f7fe640 1 -- 172.21.15.96:0/1237714516 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+2534 (secure 0 0 0) 0x7f1690017030 con 0x7f16a806f6d0 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_dump": [ 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.a", 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmAVcqLxAAIRo/psndnU3y/VG+/goNRQ==", 2024-09-13T17:43:33.935 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.b", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmpF6ANxAAzrn1jVVSCWKv5q1kdChbWA==", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:43:33.936 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.0", 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBKeeRmkXcoFhAAdX1HBKKYCHqtacYyAZbF2w==", 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.1", 2024-09-13T17:43:33.937 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBNeeRmh189GxAAFDpo1k0lxmmillsXoiKXMA==", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.2", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBQeeRmKpvLIhAA4Y7okMAZAQ9cjYpkaKF/kA==", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.938 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.0", 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRm3enLAxAAZGIwl+Bs06GY4JG0dbxLBQ==", 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow r", 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.1", 2024-09-13T17:43:33.939 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRmNlIQCRAAFFcBZ0Pwtq8kXtBAAFkuDA==", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.admin", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmUwoTGBAAycvCNfOc1ueY+QjBkpAqpA==", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:43:33.940 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.x", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmOOcEKhAAJmefIyHpVdx4zDzhwQcawA==", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:33.941 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:43:33.942 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:33.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 msgr2=0x7f16880379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 0x7f16880379e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f16980af000 tx=0x7f16980a7070 comp rx=0 tx=0).stop 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 msgr2=0x7f16a81cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f169000c3d0 tx=0x7f169000c890 comp rx=0 tx=0).stop 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 shutdown_connections 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f1688035520 0x7f16880379e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 --2- 172.21.15.96:0/1237714516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16a806f6d0 0x7f16a81cc5d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 >> 172.21.15.96:0/1237714516 conn(0x7f16a8098a60 msgr2=0x7f16a8136050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 shutdown_connections 2024-09-13T17:43:33.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:33.934+0000 7f16ad380640 1 -- 172.21.15.96:0/1237714516 wait complete. 2024-09-13T17:43:33.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf4e723640 1 Processor -- start 2024-09-13T17:43:34.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf4e723640 1 -- start start 2024-09-13T17:43:34.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf4e723640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdf48134e70 0x7fdf4809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf4e723640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4809ed60 con 0x7fdf48135250 2024-09-13T17:43:34.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf4e723640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4809ddc0 con 0x7fdf48134e70 2024-09-13T17:43:34.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf477fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdf48135250 0x7fdf48135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33752/0 (socket says 172.21.15.96:33752) 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf477fe640 1 -- 172.21.15.96:0/2962028727 learned_addr learned my addr 172.21.15.96:0/2962028727 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf47fff640 1 --2- 172.21.15.96:0/2962028727 >> v2:172.21.15.96:3300/0 conn(0x7fdf48134e70 0x7fdf4809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf46ffd640 1 -- 172.21.15.96:0/2962028727 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272778897 0 0) 0x7fdf3c001120 con 0x7fdf48135250 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf46ffd640 1 -- 172.21.15.96:0/2962028727 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf30003530 con 0x7fdf48135250 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf47fff640 1 -- 172.21.15.96:0/2962028727 >> v1:172.21.15.96:6789/0 conn(0x7fdf48135250 legacy=0x7fdf48135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.062+0000 7fdf47fff640 1 -- 172.21.15.96:0/2962028727 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480a68b0 con 0x7fdf48134e70 2024-09-13T17:43:34.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf47fff640 1 --2- 172.21.15.96:0/2962028727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4809e820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009740 tx=0x7fdf3801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4c10519859f21 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf46ffd640 1 -- 172.21.15.96:0/2962028727 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf380ab530 con 0x7fdf48134e70 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf46ffd640 1 -- 172.21.15.96:0/2962028727 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf380a7050 con 0x7fdf48134e70 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf46ffd640 1 -- 172.21.15.96:0/2962028727 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf380ab7e0 con 0x7fdf48134e70 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- 172.21.15.96:0/2962028727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 msgr2=0x7fdf4809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 --2- 172.21.15.96:0/2962028727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4809e820 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009740 tx=0x7fdf3801c420 comp rx=0 tx=0).stop 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- 172.21.15.96:0/2962028727 shutdown_connections 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 --2- 172.21.15.96:0/2962028727 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4809e820 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- 172.21.15.96:0/2962028727 >> 172.21.15.96:0/2962028727 conn(0x7fdf48098a60 msgr2=0x7fdf4809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- 172.21.15.96:0/2962028727 shutdown_connections 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- 172.21.15.96:0/2962028727 wait complete. 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 Processor -- start 2024-09-13T17:43:34.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- start start 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.066+0000 7fdf4e723640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48132f50 con 0x7fdf48134e70 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50178/0 (socket says 172.21.15.96:50178) 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf47fff640 1 -- 172.21.15.96:0/224559088 learned_addr learned my addr 172.21.15.96:0/224559088 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf47fff640 1 -- 172.21.15.96:0/224559088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf38009420 con 0x7fdf48134e70 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf47fff640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf3801a050 tx=0x7fdf380191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf380abdf0 con 0x7fdf48134e70 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf380b4410 con 0x7fdf48134e70 2024-09-13T17:43:34.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf480a1ff0 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf380b46b0 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4813ba70 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdf380b2040 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf481368b0 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 0x7fdf34037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 2587+0+0 (secure 0 0 0) 0x7fdf380e1610 con 0x7fdf48134e70 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf477fe640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 0x7fdf34037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.070+0000 7fdf477fe640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 0x7fdf34037a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001630 tx=0x7fdf3c001160 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.074+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdf380bc7c0 con 0x7fdf48134e70 2024-09-13T17:43:34.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.218+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf480a2740 con 0x7fdf48134e70 2024-09-13T17:43:34.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf44ff9640 1 -- 172.21.15.96:0/224559088 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+6723 (secure 0 0 0) 0x7fdf380b0030 con 0x7fdf48134e70 2024-09-13T17:43:34.222 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:34.223 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":33,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:33.552414+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 msgr2=0x7fdf34037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 0x7fdf34037a30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c001630 tx=0x7fdf3c001160 comp rx=0 tx=0).stop 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 msgr2=0x7fdf4813aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf3801a050 tx=0x7fdf380191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 shutdown_connections 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdf34035570 0x7fdf34037a30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 --2- 172.21.15.96:0/224559088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48134e70 0x7fdf4813aff0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 >> 172.21.15.96:0/224559088 conn(0x7fdf48098a60 msgr2=0x7fdf48098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 shutdown_connections 2024-09-13T17:43:34.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.222+0000 7fdf4e723640 1 -- 172.21.15.96:0/224559088 wait complete. 2024-09-13T17:43:34.239 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:34.240 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:34.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f77a0224640 1 Processor -- start 2024-09-13T17:43:34.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f77a0224640 1 -- start start 2024-09-13T17:43:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f77a0224640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f779806f6d0 0x7f7798134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f77a0224640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f779806e130 con 0x7f779809dc20 2024-09-13T17:43:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f77a0224640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77980a6750 con 0x7f779806f6d0 2024-09-13T17:43:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f779dfc0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f779806f6d0 0x7f7798134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f779d7bf640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f779809dc20 0x7f7798134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33758/0 (socket says 172.21.15.96:33758) 2024-09-13T17:43:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.346+0000 7f779d7bf640 1 -- 172.21.15.96:0/1135840858 learned_addr learned my addr 172.21.15.96:0/1135840858 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779cfbe640 1 -- 172.21.15.96:0/1135840858 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226255839 0 0) 0x7f779806e130 con 0x7f779809dc20 2024-09-13T17:43:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779cfbe640 1 -- 172.21.15.96:0/1135840858 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7784003530 con 0x7f779809dc20 2024-09-13T17:43:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779dfc0640 1 -- 172.21.15.96:0/1135840858 >> v1:172.21.15.96:6789/0 conn(0x7f779809dc20 legacy=0x7f7798134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.352 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779dfc0640 1 -- 172.21.15.96:0/1135840858 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980712e0 con 0x7f779806f6d0 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779dfc0640 1 --2- 172.21.15.96:0/1135840858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f7798134ab0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7794009d90 tx=0x7f779401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd931d65b497a56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779cfbe640 1 -- 172.21.15.96:0/1135840858 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77940a5640 con 0x7f779806f6d0 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779cfbe640 1 -- 172.21.15.96:0/1135840858 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940a57a0 con 0x7f779806f6d0 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f779cfbe640 1 -- 172.21.15.96:0/1135840858 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77940a5a60 con 0x7f779806f6d0 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 -- 172.21.15.96:0/1135840858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 msgr2=0x7f7798134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 --2- 172.21.15.96:0/1135840858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f7798134ab0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7794009d90 tx=0x7f779401c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 -- 172.21.15.96:0/1135840858 shutdown_connections 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 --2- 172.21.15.96:0/1135840858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f7798134ab0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 -- 172.21.15.96:0/1135840858 >> 172.21.15.96:0/1135840858 conn(0x7f7798098a60 msgr2=0x7f779809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 -- 172.21.15.96:0/1135840858 shutdown_connections 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.350+0000 7f77a0224640 1 -- 172.21.15.96:0/1135840858 wait complete. 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 Processor -- start 2024-09-13T17:43:34.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 -- start start 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77980a1fe0 con 0x7f779806f6d0 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779dfc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779dfc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50196/0 (socket says 172.21.15.96:50196) 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779dfc0640 1 -- 172.21.15.96:0/2538617161 learned_addr learned my addr 172.21.15.96:0/2538617161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779dfc0640 1 -- 172.21.15.96:0/2538617161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7794009a70 con 0x7f779806f6d0 2024-09-13T17:43:34.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779dfc0640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f779401a510 tx=0x7f7794019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7794009450 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77940ad650 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77940ad910 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77980a6110 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77980a31f0 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f77940b4050 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77980a3eb0 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 0x7f777c037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f77940e2570 con 0x7f779806f6d0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779d7bf640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 0x7f777c037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.354+0000 7f779d7bf640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 0x7f777c037a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7788007fd0 tx=0x7f7788007980 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.362+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77940c19a0 con 0x7f779806f6d0 2024-09-13T17:43:34.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.518+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f77980a1620 con 0x7f779806f6d0 2024-09-13T17:43:34.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.518+0000 7f778e7fc640 1 -- 172.21.15.96:0/2538617161 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77940b2070 con 0x7f779806f6d0 2024-09-13T17:43:34.521 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 msgr2=0x7f777c037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 0x7f777c037a30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7788007fd0 tx=0x7f7788007980 comp rx=0 tx=0).stop 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 msgr2=0x7f77980a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f779401a510 tx=0x7f7794019680 comp rx=0 tx=0).stop 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 shutdown_connections 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f777c035570 0x7f777c037a30 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 --2- 172.21.15.96:0/2538617161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f779806f6d0 0x7f77980a21d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 >> 172.21.15.96:0/2538617161 conn(0x7f7798098a60 msgr2=0x7f779812fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 shutdown_connections 2024-09-13T17:43:34.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.522+0000 7f77a0224640 1 -- 172.21.15.96:0/2538617161 wait complete. 2024-09-13T17:43:34.540 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:43:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f819070c640 1 Processor -- start 2024-09-13T17:43:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f819070c640 1 -- start start 2024-09-13T17:43:34.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f819070c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8188134e70 0x7f818809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f819070c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818809ed60 con 0x7f8188135250 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f819070c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818809ddc0 con 0x7f8188134e70 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.654+0000 7f818e4a8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8188134e70 0x7f818809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818dca7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8188135250 0x7f8188135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33764/0 (socket says 172.21.15.96:33764) 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818dca7640 1 -- 172.21.15.96:0/1130849272 learned_addr learned my addr 172.21.15.96:0/1130849272 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818e4a8640 1 -- 172.21.15.96:0/1130849272 >> v1:172.21.15.96:6789/0 conn(0x7f8188135250 legacy=0x7f8188135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818e4a8640 1 -- 172.21.15.96:0/1130849272 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81880a68b0 con 0x7f8188134e70 2024-09-13T17:43:34.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818d4a6640 1 -- 172.21.15.96:0/1130849272 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307591546 0 0) 0x7f817c001120 con 0x7f8188135250 2024-09-13T17:43:34.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818e4a8640 1 --2- 172.21.15.96:0/1130849272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818809e820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8180009d90 tx=0x7f818001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0ef40b6f363345 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818d4a6640 1 -- 172.21.15.96:0/1130849272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81800a57b0 con 0x7f8188134e70 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818d4a6640 1 -- 172.21.15.96:0/1130849272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81800a5910 con 0x7f8188134e70 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f818d4a6640 1 -- 172.21.15.96:0/1130849272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81800a5bd0 con 0x7f8188134e70 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 -- 172.21.15.96:0/1130849272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 msgr2=0x7f818809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 --2- 172.21.15.96:0/1130849272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818809e820 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8180009d90 tx=0x7f818001c970 comp rx=0 tx=0).stop 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 -- 172.21.15.96:0/1130849272 shutdown_connections 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 --2- 172.21.15.96:0/1130849272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818809e820 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 -- 172.21.15.96:0/1130849272 >> 172.21.15.96:0/1130849272 conn(0x7f8188098a60 msgr2=0x7f818809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 -- 172.21.15.96:0/1130849272 shutdown_connections 2024-09-13T17:43:34.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.658+0000 7f819070c640 1 -- 172.21.15.96:0/1130849272 wait complete. 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 Processor -- start 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 -- start start 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188070770 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818e4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818e4a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50212/0 (socket says 172.21.15.96:50212) 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818e4a8640 1 -- 172.21.15.96:0/2698877573 learned_addr learned my addr 172.21.15.96:0/2698877573 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818e4a8640 1 -- 172.21.15.96:0/2698877573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009a70 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818e4a8640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f81800b0040 tx=0x7f8180019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81800b8690 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81800b87f0 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81880a44a0 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81800b8a90 con 0x7f8188134e70 2024-09-13T17:43:34.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81880a55c0 con 0x7f8188134e70 2024-09-13T17:43:34.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f81800b4040 con 0x7f8188134e70 2024-09-13T17:43:34.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81881368b0 con 0x7f8188134e70 2024-09-13T17:43:34.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 0x7f8174037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f81800e2690 con 0x7f8188134e70 2024-09-13T17:43:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818dca7640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 0x7f8174037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.661+0000 7f818dca7640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 0x7f8174037a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c10 tx=0x7f817c001320 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.665+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f81800caba0 con 0x7f8188134e70 2024-09-13T17:43:34.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.817+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f81880a4630 con 0x7f8188134e70 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f8186ffd640 1 -- 172.21.15.96:0/2698877573 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+65 (secure 0 0 0) 0x7f81800b2070 con 0x7f8188134e70 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:34.822 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 msgr2=0x7f8174037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 0x7f8174037a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f817c000c10 tx=0x7f817c001320 comp rx=0 tx=0).stop 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 msgr2=0x7f818806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f81800b0040 tx=0x7f8180019710 comp rx=0 tx=0).stop 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 shutdown_connections 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f8174035570 0x7f8174037a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 --2- 172.21.15.96:0/2698877573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8188134e70 0x7f818806f6d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 >> 172.21.15.96:0/2698877573 conn(0x7f8188098a60 msgr2=0x7f8188098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 shutdown_connections 2024-09-13T17:43:34.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.821+0000 7f819070c640 1 -- 172.21.15.96:0/2698877573 wait complete. 2024-09-13T17:43:34.839 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:43:34.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:43:34.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8c52fd640 1 Processor -- start 2024-09-13T17:43:34.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8c52fd640 1 -- start start 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8c52fd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe8c009fd20 0x7fe8c013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8c52fd640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c009e520 con 0x7fe8c0138290 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8c52fd640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00aabe0 con 0x7fe8c009fd20 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8be7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe8c009fd20 0x7fe8c013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8beffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe8c0138290 0x7fe8c0138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33774/0 (socket says 172.21.15.96:33774) 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8beffd640 1 -- 172.21.15.96:0/1982038880 learned_addr learned my addr 172.21.15.96:0/1982038880 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:34.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8be7fc640 1 -- 172.21.15.96:0/1982038880 >> v1:172.21.15.96:6789/0 conn(0x7fe8c0138290 legacy=0x7fe8c0138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8be7fc640 1 -- 172.21.15.96:0/1982038880 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c009d610 con 0x7fe8c009fd20 2024-09-13T17:43:34.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.949+0000 7fe8bdffb640 1 -- 172.21.15.96:0/1982038880 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086118910 0 0) 0x7fe8c009e520 con 0x7fe8c0138290 2024-09-13T17:43:34.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8be7fc640 1 --2- 172.21.15.96:0/1982038880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c013d1d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009740 tx=0x7fe8a801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84965d9a455f300f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8bdffb640 1 -- 172.21.15.96:0/1982038880 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8a80ab530 con 0x7fe8c009fd20 2024-09-13T17:43:34.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8bdffb640 1 -- 172.21.15.96:0/1982038880 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a80a7050 con 0x7fe8c009fd20 2024-09-13T17:43:34.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8bdffb640 1 -- 172.21.15.96:0/1982038880 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8a80ab7e0 con 0x7fe8c009fd20 2024-09-13T17:43:34.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- 172.21.15.96:0/1982038880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 msgr2=0x7fe8c013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:34.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/1982038880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c013d1d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8009740 tx=0x7fe8a801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:34.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- 172.21.15.96:0/1982038880 shutdown_connections 2024-09-13T17:43:34.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/1982038880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c013d1d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:34.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- 172.21.15.96:0/1982038880 >> 172.21.15.96:0/1982038880 conn(0x7fe8c006c060 msgr2=0x7fe8c006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:34.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- 172.21.15.96:0/1982038880 shutdown_connections 2024-09-13T17:43:34.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- 172.21.15.96:0/1982038880 wait complete. 2024-09-13T17:43:34.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 Processor -- start 2024-09-13T17:43:34.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- start start 2024-09-13T17:43:34.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8c52fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c00a5f50 con 0x7fe8c009fd20 2024-09-13T17:43:34.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50232/0 (socket says 172.21.15.96:50232) 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.953+0000 7fe8beffd640 1 -- 172.21.15.96:0/2606667741 learned_addr learned my addr 172.21.15.96:0/2606667741 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8beffd640 1 -- 172.21.15.96:0/2606667741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8009420 con 0x7fe8c009fd20 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8beffd640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c420 tx=0x7fe8b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8b80074d0 con 0x7fe8c009fd20 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8007630 con 0x7fe8c009fd20 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c009e1a0 con 0x7fe8c009fd20 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8b8004960 con 0x7fe8c009fd20 2024-09-13T17:43:34.959 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c00a4010 con 0x7fe8c009fd20 2024-09-13T17:43:34.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0136370 con 0x7fe8c009fd20 2024-09-13T17:43:34.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fe8b8004ac0 con 0x7fe8c009fd20 2024-09-13T17:43:34.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 0x7fe8a00379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:34.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 2587+0+0 (secure 0 0 0) 0x7fe8b8046b10 con 0x7fe8c009fd20 2024-09-13T17:43:34.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8be7fc640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 0x7fe8a00379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:34.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.957+0000 7fe8be7fc640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 0x7fe8a00379e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80ad000 tx=0x7fe8a80a6000 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:34.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:34.961+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe8b801ab40 con 0x7fe8c009fd20 2024-09-13T17:43:35.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.105+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe8c0065e90 con 0x7fe8c009fd20 2024-09-13T17:43:35.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.561+0000 7fe8b77fe640 1 -- 172.21.15.96:0/2606667741 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe8b8018030 con 0x7fe8c009fd20 2024-09-13T17:43:35.565 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.569+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 msgr2=0x7fe8a00379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.569+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 0x7fe8a00379e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe8a80ad000 tx=0x7fe8a80a6000 comp rx=0 tx=0).stop 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.569+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 msgr2=0x7fe8c00a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.569+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c420 tx=0x7fe8b800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 shutdown_connections 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe8a0035520 0x7fe8a00379e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 --2- 172.21.15.96:0/2606667741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8c009fd20 0x7fe8c00a5a10 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 >> 172.21.15.96:0/2606667741 conn(0x7fe8c006c060 msgr2=0x7fe8c009a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 shutdown_connections 2024-09-13T17:43:35.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.573+0000 7fe8c52fd640 1 -- 172.21.15.96:0/2606667741 wait complete. 2024-09-13T17:43:35.596 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:43:35.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa473f640 1 Processor -- start 2024-09-13T17:43:35.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa473f640 1 -- start start 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa473f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fea9c09c180 0x7fea9c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa473f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c1326f0 con 0x7fea9c132940 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa473f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c132500 con 0x7fea9c09c180 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa24db640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fea9c132940 0x7fea9c132d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33790/0 (socket says 172.21.15.96:33790) 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa24db640 1 -- 172.21.15.96:0/1254417452 learned_addr learned my addr 172.21.15.96:0/1254417452 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa1cda640 1 --2- 172.21.15.96:0/1254417452 >> v2:172.21.15.96:3300/0 conn(0x7fea9c09c180 0x7fea9c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa14d9640 1 -- 172.21.15.96:0/1254417452 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457968746 0 0) 0x7fea9c1326f0 con 0x7fea9c132940 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa14d9640 1 -- 172.21.15.96:0/1254417452 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea88003530 con 0x7fea9c132940 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa1cda640 1 -- 172.21.15.96:0/1254417452 >> v1:172.21.15.96:6789/0 conn(0x7fea9c132940 legacy=0x7fea9c132d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.717+0000 7feaa1cda640 1 -- 172.21.15.96:0/1254417452 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c09cfd0 con 0x7fea9c09c180 2024-09-13T17:43:35.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa1cda640 1 --2- 172.21.15.96:0/1254417452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c13ae50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009740 tx=0x7fea8c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c574b024c9e0f314 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa14d9640 1 -- 172.21.15.96:0/1254417452 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea8c0ab400 con 0x7fea9c09c180 2024-09-13T17:43:35.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa14d9640 1 -- 172.21.15.96:0/1254417452 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c0a7050 con 0x7fea9c09c180 2024-09-13T17:43:35.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa14d9640 1 -- 172.21.15.96:0/1254417452 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea8c0ab6b0 con 0x7fea9c09c180 2024-09-13T17:43:35.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/1254417452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 msgr2=0x7fea9c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:35.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 --2- 172.21.15.96:0/1254417452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c13ae50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009740 tx=0x7fea8c01c390 comp rx=0 tx=0).stop 2024-09-13T17:43:35.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/1254417452 shutdown_connections 2024-09-13T17:43:35.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 --2- 172.21.15.96:0/1254417452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c13ae50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:35.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/1254417452 >> 172.21.15.96:0/1254417452 conn(0x7fea9c06e020 msgr2=0x7fea9c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:35.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/1254417452 shutdown_connections 2024-09-13T17:43:35.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/1254417452 wait complete. 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 Processor -- start 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- start start 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c0a3100 con 0x7fea9c09c180 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa24db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa24db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50252/0 (socket says 172.21.15.96:50252) 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa24db640 1 -- 172.21.15.96:0/922300011 learned_addr learned my addr 172.21.15.96:0/922300011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa24db640 1 -- 172.21.15.96:0/922300011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c009420 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa24db640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c450 tx=0x7fea9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea98005c20 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea98005d80 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9c09f0e0 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea980071a0 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.721+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c0a80c0 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fea98007300 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c0a15a0 con 0x7fea9c09c180 2024-09-13T17:43:35.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7fea92ffd640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 0x7fea80037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:35.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 2957+0+0 (secure 0 0 0) 0x7fea98046ad0 con 0x7fea9c09c180 2024-09-13T17:43:35.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7feaa1cda640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 0x7fea80037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:35.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.725+0000 7feaa1cda640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 0x7fea80037a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0ad000 tx=0x7fea8c0a6000 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:35.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.729+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea980196e0 con 0x7fea9c09c180 2024-09-13T17:43:35.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:35.885+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fea9c0a1e80 con 0x7fea9c09c180 2024-09-13T17:43:36.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.569+0000 7fea92ffd640 1 -- 172.21.15.96:0/922300011 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7fea98017070 con 0x7fea9c09c180 2024-09-13T17:43:36.575 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-13T17:43:36.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.577+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 msgr2=0x7fea80037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.577+0000 7feaa473f640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 0x7fea80037a30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0ad000 tx=0x7fea8c0a6000 comp rx=0 tx=0).stop 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.577+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 msgr2=0x7fea9c0a3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.577+0000 7feaa473f640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fea9800c450 tx=0x7fea9800c910 comp rx=0 tx=0).stop 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 shutdown_connections 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea80035570 0x7fea80037a30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 --2- 172.21.15.96:0/922300011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea9c09c180 0x7fea9c0a3850 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 >> 172.21.15.96:0/922300011 conn(0x7fea9c06e020 msgr2=0x7fea9c070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 shutdown_connections 2024-09-13T17:43:36.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.581+0000 7feaa473f640 1 -- 172.21.15.96:0/922300011 wait complete. 2024-09-13T17:43:36.616 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:43:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df2a97640 1 Processor -- start 2024-09-13T17:43:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df2a97640 1 -- start start 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df2a97640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7dec0a4900 0x7f7dec09e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df2a97640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dec071740 con 0x7f7dec0a4ce0 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df2a97640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dec09e250 con 0x7f7dec0a4900 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df0833640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7dec0a4900 0x7f7dec09e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7debfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7dec0a4ce0 0x7f7dec0a50f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33798/0 (socket says 172.21.15.96:33798) 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7debfff640 1 -- 172.21.15.96:0/1973495637 learned_addr learned my addr 172.21.15.96:0/1973495637 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df0833640 1 -- 172.21.15.96:0/1973495637 >> v1:172.21.15.96:6789/0 conn(0x7f7dec0a4ce0 legacy=0x7f7dec0a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df0833640 1 -- 172.21.15.96:0/1973495637 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dec0a5c70 con 0x7f7dec0a4900 2024-09-13T17:43:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7deb7fe640 1 -- 172.21.15.96:0/1973495637 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985537320 0 0) 0x7f7dec071740 con 0x7f7dec0a4ce0 2024-09-13T17:43:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7df0833640 1 --2- 172.21.15.96:0/1973495637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec09e9a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009740 tx=0x7f7de401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09505e34d19e095 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7deb7fe640 1 -- 172.21.15.96:0/1973495637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7de40ab530 con 0x7f7dec0a4900 2024-09-13T17:43:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7deb7fe640 1 -- 172.21.15.96:0/1973495637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40a7050 con 0x7f7dec0a4900 2024-09-13T17:43:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.733+0000 7f7deb7fe640 1 -- 172.21.15.96:0/1973495637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7de40ab7e0 con 0x7f7dec0a4900 2024-09-13T17:43:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/1973495637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 msgr2=0x7f7dec09e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 --2- 172.21.15.96:0/1973495637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec09e9a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009740 tx=0x7f7de401c390 comp rx=0 tx=0).stop 2024-09-13T17:43:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/1973495637 shutdown_connections 2024-09-13T17:43:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 --2- 172.21.15.96:0/1973495637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec09e9a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/1973495637 >> 172.21.15.96:0/1973495637 conn(0x7f7dec06a150 msgr2=0x7f7dec06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/1973495637 shutdown_connections 2024-09-13T17:43:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/1973495637 wait complete. 2024-09-13T17:43:36.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 Processor -- start 2024-09-13T17:43:36.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- start start 2024-09-13T17:43:36.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dec070f90 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df0833640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df0833640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50264/0 (socket says 172.21.15.96:50264) 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df0833640 1 -- 172.21.15.96:0/2865213042 learned_addr learned my addr 172.21.15.96:0/2865213042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df0833640 1 -- 172.21.15.96:0/2865213042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4009420 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df0833640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7de40ae040 tx=0x7f7de401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7de40b7450 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de40b75b0 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dec0706b0 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7de40b7890 con 0x7f7dec0a4900 2024-09-13T17:43:36.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dec13c5b0 con 0x7f7dec0a4900 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dec073570 con 0x7f7dec0a4900 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.737+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f7de40b79f0 con 0x7f7dec0a4900 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.741+0000 7f7de97fa640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 0x7f7dd80379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.741+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 3339+0+0 (secure 0 0 0) 0x7f7de40e21f0 con 0x7f7dec0a4900 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.741+0000 7f7debfff640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 0x7f7dd80379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.741+0000 7f7debfff640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 0x7f7dd80379e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0007920 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:36.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.745+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7de40c78f0 con 0x7f7dec0a4900 2024-09-13T17:43:36.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.937+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7dec11d270 con 0x7f7dec0a4900 2024-09-13T17:43:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 6 and data pool 7 2024-09-13T17:43:36.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.945+0000 7f7de97fa640 1 -- 172.21.15.96:0/2865213042 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v18) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7de40b00c0 con 0x7f7dec0a4900 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 msgr2=0x7f7dd80379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 0x7f7dd80379e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7de0007fd0 tx=0x7f7de0007920 comp rx=0 tx=0).stop 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 msgr2=0x7f7dec11e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7de40ae040 tx=0x7f7de401abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 shutdown_connections 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7dd8035520 0x7f7dd80379e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 --2- 172.21.15.96:0/2865213042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7dec0a4900 0x7f7dec11e600 secure :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7de40ae040 tx=0x7f7de401abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:36.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.949+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 >> 172.21.15.96:0/2865213042 conn(0x7f7dec06a150 msgr2=0x7f7dec138730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:36.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.953+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 shutdown_connections 2024-09-13T17:43:36.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:36.953+0000 7f7df2a97640 1 -- 172.21.15.96:0/2865213042 wait complete. 2024-09-13T17:43:36.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:37.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.097+0000 7f4c1f40c640 1 Processor -- start 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.097+0000 7f4c1f40c640 1 -- start start 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1f40c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c18134e70 0x7f4c1809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1f40c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1809ed60 con 0x7f4c18135250 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1f40c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1809ddc0 con 0x7f4c18134e70 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1d1a8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4c18134e70 0x7f4c1809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1c9a7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4c18135250 0x7f4c18135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33800/0 (socket says 172.21.15.96:33800) 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1c9a7640 1 -- 172.21.15.96:0/3688214184 learned_addr learned my addr 172.21.15.96:0/3688214184 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1d1a8640 1 -- 172.21.15.96:0/3688214184 >> v1:172.21.15.96:6789/0 conn(0x7f4c18135250 legacy=0x7f4c18135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1d1a8640 1 -- 172.21.15.96:0/3688214184 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c180a68b0 con 0x7f4c18134e70 2024-09-13T17:43:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c1d1a8640 1 --2- 172.21.15.96:0/3688214184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1809e820 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009740 tx=0x7f4c0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b9f17f6c57ce36 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c0ffff640 1 -- 172.21.15.96:0/3688214184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c080ab530 con 0x7f4c18134e70 2024-09-13T17:43:37.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c0ffff640 1 -- 172.21.15.96:0/3688214184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c080a7050 con 0x7f4c18134e70 2024-09-13T17:43:37.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.101+0000 7f4c0ffff640 1 -- 172.21.15.96:0/3688214184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c080ab7e0 con 0x7f4c18134e70 2024-09-13T17:43:37.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/3688214184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 msgr2=0x7f4c1809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/3688214184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1809e820 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4c08009740 tx=0x7f4c0801c420 comp rx=0 tx=0).stop 2024-09-13T17:43:37.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/3688214184 shutdown_connections 2024-09-13T17:43:37.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/3688214184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1809e820 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/3688214184 >> 172.21.15.96:0/3688214184 conn(0x7f4c18098a60 msgr2=0x7f4c1809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/3688214184 shutdown_connections 2024-09-13T17:43:37.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/3688214184 wait complete. 2024-09-13T17:43:37.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 Processor -- start 2024-09-13T17:43:37.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- start start 2024-09-13T17:43:37.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c180a1710 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1d1a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1d1a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50278/0 (socket says 172.21.15.96:50278) 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1d1a8640 1 -- 172.21.15.96:0/1151855063 learned_addr learned my addr 172.21.15.96:0/1151855063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1d1a8640 1 -- 172.21.15.96:0/1151855063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08009420 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1d1a8640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c0801a050 tx=0x7f4c080191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c080abea0 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c080b4410 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c180a3550 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c080b46b0 con 0x7f4c18134e70 2024-09-13T17:43:37.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c180a4270 con 0x7f4c18134e70 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f4c080b2040 con 0x7f4c18134e70 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 0x7f4bfc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c181368b0 con 0x7f4c18134e70 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f4c080e1aa0 con 0x7f4c18134e70 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.105+0000 7f4c1c9a7640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 0x7f4bfc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.109+0000 7f4c1c9a7640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 0x7f4bfc037a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c14001610 tx=0x7f4c14007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.113+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4c080b0070 con 0x7f4c18134e70 2024-09-13T17:43:37.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.273+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c1809b340 con 0x7f4c18134e70 2024-09-13T17:43:37.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.273+0000 7f4c0dffb640 1 -- 172.21.15.96:0/1151855063 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+10504 (secure 0 0 0) 0x7f4c080b4810 con 0x7f4c18134e70 2024-09-13T17:43:37.278 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:37.278 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":36,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:36.940254+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 msgr2=0x7f4bfc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 0x7f4bfc037a30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4c14001610 tx=0x7f4c14007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 msgr2=0x7f4c1812fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4c0801a050 tx=0x7f4c080191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 shutdown_connections 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f4bfc035570 0x7f4bfc037a30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 --2- 172.21.15.96:0/1151855063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c18134e70 0x7f4c1812fa60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 >> 172.21.15.96:0/1151855063 conn(0x7f4c18098a60 msgr2=0x7f4c18133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 shutdown_connections 2024-09-13T17:43:37.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.277+0000 7f4c1f40c640 1 -- 172.21.15.96:0/1151855063 wait complete. 2024-09-13T17:43:37.296 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:37.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e8008640 1 Processor -- start 2024-09-13T17:43:37.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e8008640 1 -- start start 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e8008640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff8e009e820 0x7ff8e013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e8008640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e00a68b0 con 0x7ff8e0134e70 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e8008640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e009ddc0 con 0x7ff8e009e820 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e55a3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff8e009e820 0x7ff8e013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e5da4640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff8e0134e70 0x7ff8e0135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33804/0 (socket says 172.21.15.96:33804) 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.409+0000 7ff8e5da4640 1 -- 172.21.15.96:0/2890791535 learned_addr learned my addr 172.21.15.96:0/2890791535 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e55a3640 1 -- 172.21.15.96:0/2890791535 >> v1:172.21.15.96:6789/0 conn(0x7ff8e0134e70 legacy=0x7ff8e0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e55a3640 1 -- 172.21.15.96:0/2890791535 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e0136970 con 0x7ff8e009e820 2024-09-13T17:43:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e55a3640 1 --2- 172.21.15.96:0/2890791535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e013ae50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009d90 tx=0x7ff8d401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a713f72503401844 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e4da2640 1 -- 172.21.15.96:0/2890791535 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8d40a57b0 con 0x7ff8e009e820 2024-09-13T17:43:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e4da2640 1 -- 172.21.15.96:0/2890791535 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d40a5910 con 0x7ff8e009e820 2024-09-13T17:43:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e4da2640 1 -- 172.21.15.96:0/2890791535 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8d40a5bd0 con 0x7ff8e009e820 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 -- 172.21.15.96:0/2890791535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 msgr2=0x7ff8e013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2890791535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e013ae50 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009d90 tx=0x7ff8d401c970 comp rx=0 tx=0).stop 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 -- 172.21.15.96:0/2890791535 shutdown_connections 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2890791535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e013ae50 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 -- 172.21.15.96:0/2890791535 >> 172.21.15.96:0/2890791535 conn(0x7ff8e0098a60 msgr2=0x7ff8e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 -- 172.21.15.96:0/2890791535 shutdown_connections 2024-09-13T17:43:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.413+0000 7ff8e8008640 1 -- 172.21.15.96:0/2890791535 wait complete. 2024-09-13T17:43:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 Processor -- start 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 -- start start 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00a3070 con 0x7ff8e009e820 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e5da4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e5da4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50306/0 (socket says 172.21.15.96:50306) 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e5da4640 1 -- 172.21.15.96:0/2009043336 learned_addr learned my addr 172.21.15.96:0/2009043336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e5da4640 1 -- 172.21.15.96:0/2009043336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4009a70 con 0x7ff8e009e820 2024-09-13T17:43:37.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e5da4640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c3a0 tx=0x7ff8d000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8d0005c20 con 0x7ff8e009e820 2024-09-13T17:43:37.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d0005d80 con 0x7ff8e009e820 2024-09-13T17:43:37.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e00a2e20 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8d0007200 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e00a17b0 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e00a31c0 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7ff8d0007360 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 0x7ff8cc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff8d0046ce0 con 0x7ff8e009e820 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e55a3640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 0x7ff8cc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.417+0000 7ff8e55a3640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 0x7ff8cc037a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40af000 tx=0x7ff8d40a7070 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.421+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8d0012400 con 0x7ff8e009e820 2024-09-13T17:43:37.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.585+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8e006a1b0 con 0x7ff8e009e820 2024-09-13T17:43:37.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.585+0000 7ff8de7fc640 1 -- 172.21.15.96:0/2009043336 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+10504 (secure 0 0 0) 0x7ff8d0017030 con 0x7ff8e009e820 2024-09-13T17:43:37.589 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:37.589 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":36,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:36.940254+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:37.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 msgr2=0x7ff8cc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 0x7ff8cc037a30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40af000 tx=0x7ff8d40a7070 comp rx=0 tx=0).stop 2024-09-13T17:43:37.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 msgr2=0x7ff8e01cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000c3a0 tx=0x7ff8d000c860 comp rx=0 tx=0).stop 2024-09-13T17:43:37.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 shutdown_connections 2024-09-13T17:43:37.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff8cc035570 0x7ff8cc037a30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 --2- 172.21.15.96:0/2009043336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8e009e820 0x7ff8e01cda00 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 >> 172.21.15.96:0/2009043336 conn(0x7ff8e0098a60 msgr2=0x7ff8e0134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 shutdown_connections 2024-09-13T17:43:37.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.589+0000 7ff8e8008640 1 -- 172.21.15.96:0/2009043336 wait complete. 2024-09-13T17:43:37.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:43:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdff04d4640 1 Processor -- start 2024-09-13T17:43:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdff04d4640 1 -- start start 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdff04d4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdfe8134e70 0x7fdfe809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdff04d4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe809ed60 con 0x7fdfe8135250 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdff04d4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe809ddc0 con 0x7fdfe8134e70 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdfeda6f640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdfe8135250 0x7fdfe8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33816/0 (socket says 172.21.15.96:33816) 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdfee270640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdfe8134e70 0x7fdfe809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdfeda6f640 1 -- 172.21.15.96:0/2924451021 learned_addr learned my addr 172.21.15.96:0/2924451021 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdfee270640 1 -- 172.21.15.96:0/2924451021 >> v1:172.21.15.96:6789/0 conn(0x7fdfe8135250 legacy=0x7fdfe8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.717+0000 7fdfee270640 1 -- 172.21.15.96:0/2924451021 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe80a68b0 con 0x7fdfe8134e70 2024-09-13T17:43:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 --2- 172.21.15.96:0/2924451021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809e820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009740 tx=0x7fdfe401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3d97e7388a2189 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfed26e640 1 -- 172.21.15.96:0/2924451021 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfe40ab530 con 0x7fdfe8134e70 2024-09-13T17:43:37.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfed26e640 1 -- 172.21.15.96:0/2924451021 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe40a7050 con 0x7fdfe8134e70 2024-09-13T17:43:37.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfed26e640 1 -- 172.21.15.96:0/2924451021 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfe40ab7e0 con 0x7fdfe8134e70 2024-09-13T17:43:37.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/2924451021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 msgr2=0x7fdfe809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 --2- 172.21.15.96:0/2924451021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809e820 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fdfe4009740 tx=0x7fdfe401c420 comp rx=0 tx=0).stop 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/2924451021 shutdown_connections 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 --2- 172.21.15.96:0/2924451021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809e820 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/2924451021 >> 172.21.15.96:0/2924451021 conn(0x7fdfe8098a60 msgr2=0x7fdfe809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/2924451021 shutdown_connections 2024-09-13T17:43:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/2924451021 wait complete. 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 Processor -- start 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- start start 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe80a5e30 con 0x7fdfe8134e70 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50318/0 (socket says 172.21.15.96:50318) 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 -- 172.21.15.96:0/1502128125 learned_addr learned my addr 172.21.15.96:0/1502128125 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 -- 172.21.15.96:0/1502128125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe4009420 con 0x7fdfe8134e70 2024-09-13T17:43:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfee270640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401a050 tx=0x7fdfe40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfe40abea0 con 0x7fdfe8134e70 2024-09-13T17:43:37.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfe40b4410 con 0x7fdfe8134e70 2024-09-13T17:43:37.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe80a23f0 con 0x7fdfe8134e70 2024-09-13T17:43:37.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdfe40b46b0 con 0x7fdfe8134e70 2024-09-13T17:43:37.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.721+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe80a0890 con 0x7fdfe8134e70 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdfe40b2040 con 0x7fdfe8134e70 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe80a20e0 con 0x7fdfe8134e70 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdfdeffd640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 0x7fdfcc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fdfe40e1a70 con 0x7fdfe8134e70 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdfeda6f640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 0x7fdfcc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:37.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.725+0000 7fdfeda6f640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 0x7fdfcc037a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001610 tx=0x7fdfd8007ad0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:37.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.729+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdfe40e3e60 con 0x7fdfe8134e70 2024-09-13T17:43:37.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.917+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdfe81368b0 con 0x7fdfe8134e70 2024-09-13T17:43:37.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.945+0000 7fdfdeffd640 1 -- 172.21.15.96:0/1502128125 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdfe40b0030 con 0x7fdfe8134e70 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 msgr2=0x7fdfcc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 0x7fdfcc037a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdfd8001610 tx=0x7fdfd8007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 msgr2=0x7fdfe809f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fdfe401a050 tx=0x7fdfe40191c0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 shutdown_connections 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdfcc035570 0x7fdfcc037a30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 --2- 172.21.15.96:0/1502128125 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfe8134e70 0x7fdfe809f280 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 >> 172.21.15.96:0/1502128125 conn(0x7fdfe8098a60 msgr2=0x7fdfe80995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 shutdown_connections 2024-09-13T17:43:37.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:37.949+0000 7fdff04d4640 1 -- 172.21.15.96:0/1502128125 wait complete. 2024-09-13T17:43:37.968 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:43:37.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:43:38.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.085+0000 7f21e9c57640 1 Processor -- start 2024-09-13T17:43:38.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e9c57640 1 -- start start 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e9c57640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f21e41381b0 0x7f21e40a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e9c57640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e406ca30 con 0x7f21e4138590 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e9c57640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40a02e0 con 0x7f21e41381b0 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f21e41381b0 0x7f21e40a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e37fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f21e41381b0 0x7f21e40a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50322/0 (socket says 172.21.15.96:50322) 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e2ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f21e4138590 0x7f21e406b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33832/0 (socket says 172.21.15.96:33832) 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e37fe640 1 -- 172.21.15.96:0/2604639975 learned_addr learned my addr 172.21.15.96:0/2604639975 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e27fc640 1 -- 172.21.15.96:0/2604639975 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875859782 0 0) 0x7f21e406ca30 con 0x7f21e4138590 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e27fc640 1 -- 172.21.15.96:0/2604639975 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21cc003530 con 0x7f21e4138590 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e37fe640 1 -- 172.21.15.96:0/2604639975 >> v1:172.21.15.96:6789/0 conn(0x7f21e4138590 legacy=0x7f21e406b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.089+0000 7f21e37fe640 1 -- 172.21.15.96:0/2604639975 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e40a05a0 con 0x7f21e41381b0 2024-09-13T17:43:38.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 --2- 172.21.15.96:0/2604639975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a0a30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f21d40097f0 tx=0x7f21d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffb0e980f0a45ab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e27fc640 1 -- 172.21.15.96:0/2604639975 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d40ad460 con 0x7f21e41381b0 2024-09-13T17:43:38.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e27fc640 1 -- 172.21.15.96:0/2604639975 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d40ad5c0 con 0x7f21e41381b0 2024-09-13T17:43:38.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e27fc640 1 -- 172.21.15.96:0/2604639975 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d40ad750 con 0x7f21e41381b0 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/2604639975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 msgr2=0x7f21e40a0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 --2- 172.21.15.96:0/2604639975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a0a30 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f21d40097f0 tx=0x7f21d401c450 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/2604639975 shutdown_connections 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 --2- 172.21.15.96:0/2604639975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a0a30 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/2604639975 >> 172.21.15.96:0/2604639975 conn(0x7f21e406ef80 msgr2=0x7f21e40713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/2604639975 shutdown_connections 2024-09-13T17:43:38.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/2604639975 wait complete. 2024-09-13T17:43:38.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 Processor -- start 2024-09-13T17:43:38.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- start start 2024-09-13T17:43:38.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40a4490 con 0x7f21e41381b0 2024-09-13T17:43:38.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50332/0 (socket says 172.21.15.96:50332) 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 -- 172.21.15.96:0/4200231206 learned_addr learned my addr 172.21.15.96:0/4200231206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 -- 172.21.15.96:0/4200231206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40094d0 con 0x7f21e41381b0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e37fe640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f21d40b0040 tx=0x7f21d40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d40b6440 con 0x7f21e41381b0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d40b65a0 con 0x7f21e41381b0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40a6260 con 0x7f21e41381b0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21d40b6840 con 0x7f21e41381b0 2024-09-13T17:43:38.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.093+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e413d030 con 0x7f21e41381b0 2024-09-13T17:43:38.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.097+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f21d40b4070 con 0x7f21e41381b0 2024-09-13T17:43:38.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.097+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40a0a30 con 0x7f21e41381b0 2024-09-13T17:43:38.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.097+0000 7f21cbfff640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 0x7f21d0037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:38.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.097+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f21d40e19a0 con 0x7f21e41381b0 2024-09-13T17:43:38.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.097+0000 7f21e2ffd640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 0x7f21d0037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:38.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.101+0000 7f21e2ffd640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 0x7f21d0037a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21d8007fd0 tx=0x7f21d8007940 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:38.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.101+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f21d40b4500 con 0x7f21e41381b0 2024-09-13T17:43:38.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.281+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f21e406d080 con 0x7f21e41381b0 2024-09-13T17:43:38.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.945+0000 7f21cbfff640 1 -- 172.21.15.96:0/4200231206 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21d40ba020 con 0x7f21e41381b0 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.949+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 msgr2=0x7f21d0037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.949+0000 7f21e9c57640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 0x7f21d0037a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f21d8007fd0 tx=0x7f21d8007940 comp rx=0 tx=0).stop 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.949+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 msgr2=0x7f21e40a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.949+0000 7f21e9c57640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f21d40b0040 tx=0x7f21d40191f0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 shutdown_connections 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f21d0035570 0x7f21d0037a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 --2- 172.21.15.96:0/4200231206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f21e41381b0 0x7f21e40a69b0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:38.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 >> 172.21.15.96:0/4200231206 conn(0x7f21e406ef80 msgr2=0x7f21e41350f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 shutdown_connections 2024-09-13T17:43:38.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:38.953+0000 7f21e9c57640 1 -- 172.21.15.96:0/4200231206 wait complete. 2024-09-13T17:43:38.974 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:43:38.974 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.097+0000 7f7478860640 1 Processor -- start 2024-09-13T17:43:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.097+0000 7f7478860640 1 -- start start 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.097+0000 7f7478860640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f747409d690 0x7f7474138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.097+0000 7f7478860640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7474138050 con 0x7f74741387d0 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.097+0000 7f7478860640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747409c410 con 0x7f747409d690 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f747409d690 0x7f7474138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7471d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f74741387d0 0x7f7474134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33842/0 (socket says 172.21.15.96:33842) 2024-09-13T17:43:39.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7471d9b640 1 -- 172.21.15.96:0/1285627683 learned_addr learned my addr 172.21.15.96:0/1285627683 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747259c640 1 -- 172.21.15.96:0/1285627683 >> v1:172.21.15.96:6789/0 conn(0x7f74741387d0 legacy=0x7f7474134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747159a640 1 -- 172.21.15.96:0/1285627683 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1414958758 0 0) 0x7f745c0014a0 con 0x7f74741387d0 2024-09-13T17:43:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747259c640 1 -- 172.21.15.96:0/1285627683 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747409cba0 con 0x7f747409d690 2024-09-13T17:43:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747259c640 1 --2- 172.21.15.96:0/1285627683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f7474138290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7464009740 tx=0x7f746401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e97ae8b98d7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747159a640 1 -- 172.21.15.96:0/1285627683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74640a8840 con 0x7f747409d690 2024-09-13T17:43:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747159a640 1 -- 172.21.15.96:0/1285627683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640a89a0 con 0x7f747409d690 2024-09-13T17:43:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f747159a640 1 -- 172.21.15.96:0/1285627683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74640a8c60 con 0x7f747409d690 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 -- 172.21.15.96:0/1285627683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 msgr2=0x7f7474138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 --2- 172.21.15.96:0/1285627683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f7474138290 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f7464009740 tx=0x7f746401ec10 comp rx=0 tx=0).stop 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 -- 172.21.15.96:0/1285627683 shutdown_connections 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 --2- 172.21.15.96:0/1285627683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f7474138290 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 -- 172.21.15.96:0/1285627683 >> 172.21.15.96:0/1285627683 conn(0x7f747406bc60 msgr2=0x7f747406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 -- 172.21.15.96:0/1285627683 shutdown_connections 2024-09-13T17:43:39.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.101+0000 7f7478860640 1 -- 172.21.15.96:0/1285627683 wait complete. 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 Processor -- start 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 -- start start 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74740a61a0 con 0x7f747409d690 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f747259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f747259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50348/0 (socket says 172.21.15.96:50348) 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f747259c640 1 -- 172.21.15.96:0/171832679 learned_addr learned my addr 172.21.15.96:0/171832679 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f747259c640 1 -- 172.21.15.96:0/171832679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7464009420 con 0x7f747409d690 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f747259c640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f746401b820 tx=0x7f746401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7464004150 con 0x7f747409d690 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640042b0 con 0x7f747409d690 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74740a5ee0 con 0x7f747409d690 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74640bc720 con 0x7f747409d690 2024-09-13T17:43:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747409f420 con 0x7f747409d690 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747409bc00 con 0x7f747409d690 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f74640bc880 con 0x7f747409d690 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 0x7f7454037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f74640e5960 con 0x7f747409d690 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7471d9b640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 0x7f7454037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.105+0000 7f7471d9b640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 0x7f7454037a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f745c000f90 tx=0x7f745c0015e0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.109+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f747409bc00 con 0x7f747409d690 2024-09-13T17:43:39.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74740a62e0 con 0x7f747409d690 2024-09-13T17:43:39.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f746affd640 1 -- 172.21.15.96:0/171832679 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3217 (secure 0 0 0) 0x7f74740a62e0 con 0x7f747409d690 2024-09-13T17:43:39.292 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:39.292 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T17:43:39.292 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 20 2024-09-13T17:43:39.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 msgr2=0x7f7454037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 0x7f7454037a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f745c000f90 tx=0x7f745c0015e0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 msgr2=0x7f74740a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f746401b820 tx=0x7f746401d450 comp rx=0 tx=0).stop 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 shutdown_connections 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7454035570 0x7f7454037a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 --2- 172.21.15.96:0/171832679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f747409d690 0x7f74740a6920 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 >> 172.21.15.96:0/171832679 conn(0x7f747406bc60 msgr2=0x7f747409e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 shutdown_connections 2024-09-13T17:43:39.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.289+0000 7f7478860640 1 -- 172.21.15.96:0/171832679 wait complete. 2024-09-13T17:43:39.308 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:39.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.409+0000 7f0701c28640 1 Processor -- start 2024-09-13T17:43:39.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.409+0000 7f0701c28640 1 -- start start 2024-09-13T17:43:39.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f06fc135270 0x7f06fc13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc070de0 con 0x7f06fc134e90 2024-09-13T17:43:39.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc071d60 con 0x7f06fc135270 2024-09-13T17:43:39.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06faffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f06fc135270 0x7f06fc13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06fb7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f06fc134e90 0x7f06fc071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33846/0 (socket says 172.21.15.96:33846) 2024-09-13T17:43:39.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06fb7fe640 1 -- 172.21.15.96:0/2492802420 learned_addr learned my addr 172.21.15.96:0/2492802420 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:39.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06faffd640 1 -- 172.21.15.96:0/2492802420 >> v1:172.21.15.96:6789/0 conn(0x7f06fc134e90 legacy=0x7f06fc071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06faffd640 1 -- 172.21.15.96:0/2492802420 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc06f110 con 0x7f06fc135270 2024-09-13T17:43:39.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06faffd640 1 --2- 172.21.15.96:0/2492802420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc135270 0x7f06fc13cfe0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009740 tx=0x7f06f001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51eedb276931ea8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06fa7fc640 1 -- 172.21.15.96:0/2492802420 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f00ab530 con 0x7f06fc135270 2024-09-13T17:43:39.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06fa7fc640 1 -- 172.21.15.96:0/2492802420 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f00a7050 con 0x7f06fc135270 2024-09-13T17:43:39.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f06fa7fc640 1 -- 172.21.15.96:0/2492802420 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06f00ab7e0 con 0x7f06fc135270 2024-09-13T17:43:39.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- 172.21.15.96:0/2492802420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc135270 msgr2=0x7f06fc13cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 --2- 172.21.15.96:0/2492802420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc135270 0x7f06fc13cfe0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f06f0009740 tx=0x7f06f001c390 comp rx=0 tx=0).stop 2024-09-13T17:43:39.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- 172.21.15.96:0/2492802420 shutdown_connections 2024-09-13T17:43:39.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 --2- 172.21.15.96:0/2492802420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc135270 0x7f06fc13cfe0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- 172.21.15.96:0/2492802420 >> 172.21.15.96:0/2492802420 conn(0x7f06fc098a60 msgr2=0x7f06fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- 172.21.15.96:0/2492802420 shutdown_connections 2024-09-13T17:43:39.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.413+0000 7f0701c28640 1 -- 172.21.15.96:0/2492802420 wait complete. 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 Processor -- start 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 -- start start 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0a3700 con 0x7f06fc134e90 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06fb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06fb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50362/0 (socket says 172.21.15.96:50362) 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06fb7fe640 1 -- 172.21.15.96:0/952239964 learned_addr learned my addr 172.21.15.96:0/952239964 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06fb7fe640 1 -- 172.21.15.96:0/952239964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f0009420 con 0x7f06fc134e90 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06fb7fe640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c480 tx=0x7f06ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06ec005c20 con 0x7f06fc134e90 2024-09-13T17:43:39.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ec005d80 con 0x7f06fc134e90 2024-09-13T17:43:39.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f06ec0071a0 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc0a1750 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc136cd0 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc071530 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f06ec007300 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f06ec046ce0 con 0x7f06fc134e90 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06faffd640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.417+0000 7f06faffd640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 0x7f06e8037a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ad000 tx=0x7f06f00a6000 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.421+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f06ec01b3b0 con 0x7f06fc134e90 2024-09-13T17:43:39.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.569+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06fc0a3480 con 0x7f06fc134e90 2024-09-13T17:43:39.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.569+0000 7f06e3fff640 1 -- 172.21.15.96:0/952239964 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f06ec017070 con 0x7f06fc134e90 2024-09-13T17:43:39.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:39.573 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":37,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:37.942012+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 msgr2=0x7f06e8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 0x7f06e8037a30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f06f00ad000 tx=0x7f06f00a6000 comp rx=0 tx=0).stop 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 msgr2=0x7f06fc0a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f06ec00c480 tx=0x7f06ec00c940 comp rx=0 tx=0).stop 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 shutdown_connections 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f06e8035570 0x7f06e8037a30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 --2- 172.21.15.96:0/952239964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f06fc134e90 0x7f06fc0a4460 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 >> 172.21.15.96:0/952239964 conn(0x7f06fc098a60 msgr2=0x7f06fc133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 shutdown_connections 2024-09-13T17:43:39.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.573+0000 7f0701c28640 1 -- 172.21.15.96:0/952239964 wait complete. 2024-09-13T17:43:39.590 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:43:39.590 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:43:39.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff68ac57640 1 Processor -- start 2024-09-13T17:43:39.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff68ac57640 1 -- start start 2024-09-13T17:43:39.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff68ac57640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff684134e90 0x7ff684071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff68ac57640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff684070de0 con 0x7ff684135270 2024-09-13T17:43:39.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff68ac57640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff684071d60 con 0x7ff684134e90 2024-09-13T17:43:39.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff683fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff684135270 0x7ff68406dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33850/0 (socket says 172.21.15.96:33850) 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff6889f3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff684134e90 0x7ff684071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.705+0000 7ff683fff640 1 -- 172.21.15.96:0/944599969 learned_addr learned my addr 172.21.15.96:0/944599969 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6837fe640 1 -- 172.21.15.96:0/944599969 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845296109 0 0) 0x7ff684070de0 con 0x7ff684135270 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6837fe640 1 -- 172.21.15.96:0/944599969 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c003530 con 0x7ff684135270 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6889f3640 1 -- 172.21.15.96:0/944599969 >> v1:172.21.15.96:6789/0 conn(0x7ff684135270 legacy=0x7ff68406dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6889f3640 1 -- 172.21.15.96:0/944599969 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68406f110 con 0x7ff684134e90 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6889f3640 1 --2- 172.21.15.96:0/944599969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff684071530 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff678009740 tx=0x7ff67801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deeac70e93bdac7c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6837fe640 1 -- 172.21.15.96:0/944599969 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780ab530 con 0x7ff684134e90 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6837fe640 1 -- 172.21.15.96:0/944599969 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780a7050 con 0x7ff684134e90 2024-09-13T17:43:39.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff6837fe640 1 -- 172.21.15.96:0/944599969 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780ab7e0 con 0x7ff684134e90 2024-09-13T17:43:39.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 -- 172.21.15.96:0/944599969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 msgr2=0x7ff684071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 --2- 172.21.15.96:0/944599969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff684071530 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff678009740 tx=0x7ff67801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:39.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 -- 172.21.15.96:0/944599969 shutdown_connections 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 --2- 172.21.15.96:0/944599969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff684071530 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 -- 172.21.15.96:0/944599969 >> 172.21.15.96:0/944599969 conn(0x7ff684098a60 msgr2=0x7ff68409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 -- 172.21.15.96:0/944599969 shutdown_connections 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.709+0000 7ff68ac57640 1 -- 172.21.15.96:0/944599969 wait complete. 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 Processor -- start 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 -- start start 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840a4e80 con 0x7ff684134e90 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6889f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6889f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50380/0 (socket says 172.21.15.96:50380) 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6889f3640 1 -- 172.21.15.96:0/833291240 learned_addr learned my addr 172.21.15.96:0/833291240 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:39.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6889f3640 1 -- 172.21.15.96:0/833291240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff678009420 con 0x7ff684134e90 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6889f3640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff678019fc0 tx=0x7ff67801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780abe10 con 0x7ff684134e90 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6780b7450 con 0x7ff684134e90 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68413d530 con 0x7ff684134e90 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6780b76f0 con 0x7ff684134e90 2024-09-13T17:43:39.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6840a62d0 con 0x7ff684134e90 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7ff6780b7850 con 0x7ff684134e90 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68411c930 con 0x7ff684134e90 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 0x7ff670037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff6780e24e0 con 0x7ff684134e90 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff683fff640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 0x7ff670037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:39.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.713+0000 7ff683fff640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 0x7ff670037a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff674007fd0 tx=0x7ff674007980 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.717+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff6780c76c0 con 0x7ff684134e90 2024-09-13T17:43:39.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.873+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 --> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff68413d000 con 0x7ff670035570 2024-09-13T17:43:39.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.881+0000 7ff6817fa640 1 -- 172.21.15.96:0/833291240 <== mgr.4630 v2:172.21.15.96:6824/526856805 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249937 (secure 0 0 0) 0x7ff68413d000 con 0x7ff670035570 2024-09-13T17:43:39.886 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:43:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 msgr2=0x7ff670037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 0x7ff670037a30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff674007fd0 tx=0x7ff674007980 comp rx=0 tx=0).stop 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 msgr2=0x7ff6840a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff678019fc0 tx=0x7ff67801abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 shutdown_connections 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7ff670035570 0x7ff670037a30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 --2- 172.21.15.96:0/833291240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff684134e90 0x7ff6840a5390 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 >> 172.21.15.96:0/833291240 conn(0x7ff684098a60 msgr2=0x7ff684133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 shutdown_connections 2024-09-13T17:43:39.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:39.885+0000 7ff68ac57640 1 -- 172.21.15.96:0/833291240 wait complete. 2024-09-13T17:43:39.904 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T17:43:39.050377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":221,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":91440,"kb_used_data":2176,"kb_used_omap":4,"kb_used_meta":89211,"kb_avail":283024080,"statfs":{"total":289910292480,"available":289816657920,"internally_reserved":0,"allocated":2228224,"data_stored":1551843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":91352403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000825"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273887+0000","last_change":"2024-09-13T17:43:36.972254+0000","last_active":"2024-09-13T17:43:38.273887+0000","last_peered":"2024-09-13T17:43:38.273887+0000","last_clean":"2024-09-13T17:43:38.273887+0000","last_became_active":"2024-09-13T17:43:36.971289+0000","last_became_peered":"2024-09-13T17:43:36.971289+0000","last_unstale":"2024-09-13T17:43:38.273887+0000","last_undegraded":"2024-09-13T17:43:38.273887+0000","last_fullsized":"2024-09-13T17:43:38.273887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:09.542526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321750+0000","last_change":"2024-09-13T17:43:36.974086+0000","last_active":"2024-09-13T17:43:38.321750+0000","last_peered":"2024-09-13T17:43:38.321750+0000","last_clean":"2024-09-13T17:43:38.321750+0000","last_became_active":"2024-09-13T17:43:36.973536+0000","last_became_peered":"2024-09-13T17:43:36.973536+0000","last_unstale":"2024-09-13T17:43:38.321750+0000","last_undegraded":"2024-09-13T17:43:38.321750+0000","last_fullsized":"2024-09-13T17:43:38.321750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:22:24.478487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951430+0000","last_change":"2024-09-13T17:43:36.988922+0000","last_active":"2024-09-13T17:43:37.951430+0000","last_peered":"2024-09-13T17:43:37.951430+0000","last_clean":"2024-09-13T17:43:37.951430+0000","last_became_active":"2024-09-13T17:43:36.988648+0000","last_became_peered":"2024-09-13T17:43:36.988648+0000","last_unstale":"2024-09-13T17:43:37.951430+0000","last_undegraded":"2024-09-13T17:43:37.951430+0000","last_fullsized":"2024-09-13T17:43:37.951430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273031+0000","last_change":"2024-09-13T17:43:36.972979+0000","last_active":"2024-09-13T17:43:38.273031+0000","last_peered":"2024-09-13T17:43:38.273031+0000","last_clean":"2024-09-13T17:43:38.273031+0000","last_became_active":"2024-09-13T17:43:36.972625+0000","last_became_peered":"2024-09-13T17:43:36.972625+0000","last_unstale":"2024-09-13T17:43:38.273031+0000","last_undegraded":"2024-09-13T17:43:38.273031+0000","last_fullsized":"2024-09-13T17:43:38.273031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:52.901570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272355+0000","last_change":"2024-09-13T17:43:36.998726+0000","last_active":"2024-09-13T17:43:38.272355+0000","last_peered":"2024-09-13T17:43:38.272355+0000","last_clean":"2024-09-13T17:43:38.272355+0000","last_became_active":"2024-09-13T17:43:36.998283+0000","last_became_peered":"2024-09-13T17:43:36.998283+0000","last_unstale":"2024-09-13T17:43:38.272355+0000","last_undegraded":"2024-09-13T17:43:38.272355+0000","last_fullsized":"2024-09-13T17:43:38.272355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:03.727034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272105+0000","last_change":"2024-09-13T17:43:36.996807+0000","last_active":"2024-09-13T17:43:38.272105+0000","last_peered":"2024-09-13T17:43:38.272105+0000","last_clean":"2024-09-13T17:43:38.272105+0000","last_became_active":"2024-09-13T17:43:36.996515+0000","last_became_peered":"2024-09-13T17:43:36.996515+0000","last_unstale":"2024-09-13T17:43:38.272105+0000","last_undegraded":"2024-09-13T17:43:38.272105+0000","last_fullsized":"2024-09-13T17:43:38.272105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:46.786049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272149+0000","last_change":"2024-09-13T17:43:36.999611+0000","last_active":"2024-09-13T17:43:38.272149+0000","last_peered":"2024-09-13T17:43:38.272149+0000","last_clean":"2024-09-13T17:43:38.272149+0000","last_became_active":"2024-09-13T17:43:36.998996+0000","last_became_peered":"2024-09-13T17:43:36.998996+0000","last_unstale":"2024-09-13T17:43:38.272149+0000","last_undegraded":"2024-09-13T17:43:38.272149+0000","last_fullsized":"2024-09-13T17:43:38.272149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:40.880802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271469+0000","last_change":"2024-09-13T17:43:36.997792+0000","last_active":"2024-09-13T17:43:38.271469+0000","last_peered":"2024-09-13T17:43:38.271469+0000","last_clean":"2024-09-13T17:43:38.271469+0000","last_became_active":"2024-09-13T17:43:36.997361+0000","last_became_peered":"2024-09-13T17:43:36.997361+0000","last_unstale":"2024-09-13T17:43:38.271469+0000","last_undegraded":"2024-09-13T17:43:38.271469+0000","last_fullsized":"2024-09-13T17:43:38.271469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:40.506501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274050+0000","last_change":"2024-09-13T17:43:36.998418+0000","last_active":"2024-09-13T17:43:38.274050+0000","last_peered":"2024-09-13T17:43:38.274050+0000","last_clean":"2024-09-13T17:43:38.274050+0000","last_became_active":"2024-09-13T17:43:36.998145+0000","last_became_peered":"2024-09-13T17:43:36.998145+0000","last_unstale":"2024-09-13T17:43:38.274050+0000","last_undegraded":"2024-09-13T17:43:38.274050+0000","last_fullsized":"2024-09-13T17:43:38.274050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:12.103252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273758+0000","last_change":"2024-09-13T17:43:36.972143+0000","last_active":"2024-09-13T17:43:38.273758+0000","last_peered":"2024-09-13T17:43:38.273758+0000","last_clean":"2024-09-13T17:43:38.273758+0000","last_became_active":"2024-09-13T17:43:36.971262+0000","last_became_peered":"2024-09-13T17:43:36.971262+0000","last_unstale":"2024-09-13T17:43:38.273758+0000","last_undegraded":"2024-09-13T17:43:38.273758+0000","last_fullsized":"2024-09-13T17:43:38.273758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:24.359313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273332+0000","last_change":"2024-09-13T17:43:36.999259+0000","last_active":"2024-09-13T17:43:38.273332+0000","last_peered":"2024-09-13T17:43:38.273332+0000","last_clean":"2024-09-13T17:43:38.273332+0000","last_became_active":"2024-09-13T17:43:36.998970+0000","last_became_peered":"2024-09-13T17:43:36.998970+0000","last_unstale":"2024-09-13T17:43:38.273332+0000","last_undegraded":"2024-09-13T17:43:38.273332+0000","last_fullsized":"2024-09-13T17:43:38.273332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:26.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321316+0000","last_change":"2024-09-13T17:43:36.970077+0000","last_active":"2024-09-13T17:43:38.321316+0000","last_peered":"2024-09-13T17:43:38.321316+0000","last_clean":"2024-09-13T17:43:38.321316+0000","last_became_active":"2024-09-13T17:43:36.969731+0000","last_became_peered":"2024-09-13T17:43:36.969731+0000","last_unstale":"2024-09-13T17:43:38.321316+0000","last_undegraded":"2024-09-13T17:43:38.321316+0000","last_fullsized":"2024-09-13T17:43:38.321316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321054+0000","last_change":"2024-09-13T17:43:36.989784+0000","last_active":"2024-09-13T17:43:38.321054+0000","last_peered":"2024-09-13T17:43:38.321054+0000","last_clean":"2024-09-13T17:43:38.321054+0000","last_became_active":"2024-09-13T17:43:36.989513+0000","last_became_peered":"2024-09-13T17:43:36.989513+0000","last_unstale":"2024-09-13T17:43:38.321054+0000","last_undegraded":"2024-09-13T17:43:38.321054+0000","last_fullsized":"2024-09-13T17:43:38.321054+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272297+0000","last_change":"2024-09-13T17:43:36.998495+0000","last_active":"2024-09-13T17:43:38.272297+0000","last_peered":"2024-09-13T17:43:38.272297+0000","last_clean":"2024-09-13T17:43:38.272297+0000","last_became_active":"2024-09-13T17:43:36.998203+0000","last_became_peered":"2024-09-13T17:43:36.998203+0000","last_unstale":"2024-09-13T17:43:38.272297+0000","last_undegraded":"2024-09-13T17:43:38.272297+0000","last_fullsized":"2024-09-13T17:43:38.272297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:39.769029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320313+0000","last_change":"2024-09-13T17:43:36.989971+0000","last_active":"2024-09-13T17:43:38.320313+0000","last_peered":"2024-09-13T17:43:38.320313+0000","last_clean":"2024-09-13T17:43:38.320313+0000","last_became_active":"2024-09-13T17:43:36.989598+0000","last_became_peered":"2024-09-13T17:43:36.989598+0000","last_unstale":"2024-09-13T17:43:38.320313+0000","last_undegraded":"2024-09-13T17:43:38.320313+0000","last_fullsized":"2024-09-13T17:43:38.320313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271593+0000","last_change":"2024-09-13T17:43:36.997635+0000","last_active":"2024-09-13T17:43:38.271593+0000","last_peered":"2024-09-13T17:43:38.271593+0000","last_clean":"2024-09-13T17:43:38.271593+0000","last_became_active":"2024-09-13T17:43:36.997014+0000","last_became_peered":"2024-09-13T17:43:36.997014+0000","last_unstale":"2024-09-13T17:43:38.271593+0000","last_undegraded":"2024-09-13T17:43:38.271593+0000","last_fullsized":"2024-09-13T17:43:38.271593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:36.971585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322018+0000","last_change":"2024-09-13T17:43:36.976795+0000","last_active":"2024-09-13T17:43:38.322018+0000","last_peered":"2024-09-13T17:43:38.322018+0000","last_clean":"2024-09-13T17:43:38.322018+0000","last_became_active":"2024-09-13T17:43:36.974036+0000","last_became_peered":"2024-09-13T17:43:36.974036+0000","last_unstale":"2024-09-13T17:43:38.322018+0000","last_undegraded":"2024-09-13T17:43:38.322018+0000","last_fullsized":"2024-09-13T17:43:38.322018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951220+0000","last_change":"2024-09-13T17:43:36.991727+0000","last_active":"2024-09-13T17:43:37.951220+0000","last_peered":"2024-09-13T17:43:37.951220+0000","last_clean":"2024-09-13T17:43:37.951220+0000","last_became_active":"2024-09-13T17:43:36.988498+0000","last_became_peered":"2024-09-13T17:43:36.988498+0000","last_unstale":"2024-09-13T17:43:37.951220+0000","last_undegraded":"2024-09-13T17:43:37.951220+0000","last_fullsized":"2024-09-13T17:43:37.951220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273424+0000","last_change":"2024-09-13T17:43:36.996728+0000","last_active":"2024-09-13T17:43:38.273424+0000","last_peered":"2024-09-13T17:43:38.273424+0000","last_clean":"2024-09-13T17:43:38.273424+0000","last_became_active":"2024-09-13T17:43:36.996468+0000","last_became_peered":"2024-09-13T17:43:36.996468+0000","last_unstale":"2024-09-13T17:43:38.273424+0000","last_undegraded":"2024-09-13T17:43:38.273424+0000","last_fullsized":"2024-09-13T17:43:38.273424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:56.456764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321323+0000","last_change":"2024-09-13T17:43:36.976924+0000","last_active":"2024-09-13T17:43:38.321323+0000","last_peered":"2024-09-13T17:43:38.321323+0000","last_clean":"2024-09-13T17:43:38.321323+0000","last_became_active":"2024-09-13T17:43:36.975597+0000","last_became_peered":"2024-09-13T17:43:36.975597+0000","last_unstale":"2024-09-13T17:43:38.321323+0000","last_undegraded":"2024-09-13T17:43:38.321323+0000","last_fullsized":"2024-09-13T17:43:38.321323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:43.031842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321204+0000","last_change":"2024-09-13T17:43:36.981632+0000","last_active":"2024-09-13T17:43:38.321204+0000","last_peered":"2024-09-13T17:43:38.321204+0000","last_clean":"2024-09-13T17:43:38.321204+0000","last_became_active":"2024-09-13T17:43:36.977775+0000","last_became_peered":"2024-09-13T17:43:36.977775+0000","last_unstale":"2024-09-13T17:43:38.321204+0000","last_undegraded":"2024-09-13T17:43:38.321204+0000","last_fullsized":"2024-09-13T17:43:38.321204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272348+0000","last_change":"2024-09-13T17:43:36.995318+0000","last_active":"2024-09-13T17:43:38.272348+0000","last_peered":"2024-09-13T17:43:38.272348+0000","last_clean":"2024-09-13T17:43:38.272348+0000","last_became_active":"2024-09-13T17:43:36.995041+0000","last_became_peered":"2024-09-13T17:43:36.995041+0000","last_unstale":"2024-09-13T17:43:38.272348+0000","last_undegraded":"2024-09-13T17:43:38.272348+0000","last_fullsized":"2024-09-13T17:43:38.272348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:59.884186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271912+0000","last_change":"2024-09-13T17:43:36.972925+0000","last_active":"2024-09-13T17:43:38.271912+0000","last_peered":"2024-09-13T17:43:38.271912+0000","last_clean":"2024-09-13T17:43:38.271912+0000","last_became_active":"2024-09-13T17:43:36.972388+0000","last_became_peered":"2024-09-13T17:43:36.972388+0000","last_unstale":"2024-09-13T17:43:38.271912+0000","last_undegraded":"2024-09-13T17:43:38.271912+0000","last_fullsized":"2024-09-13T17:43:38.271912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:30.822850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320627+0000","last_change":"2024-09-13T17:43:36.989729+0000","last_active":"2024-09-13T17:43:38.320627+0000","last_peered":"2024-09-13T17:43:38.320627+0000","last_clean":"2024-09-13T17:43:38.320627+0000","last_became_active":"2024-09-13T17:43:36.989287+0000","last_became_peered":"2024-09-13T17:43:36.989287+0000","last_unstale":"2024-09-13T17:43:38.320627+0000","last_undegraded":"2024-09-13T17:43:38.320627+0000","last_fullsized":"2024-09-13T17:43:38.320627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274099+0000","last_change":"2024-09-13T17:43:36.972052+0000","last_active":"2024-09-13T17:43:38.274099+0000","last_peered":"2024-09-13T17:43:38.274099+0000","last_clean":"2024-09-13T17:43:38.274099+0000","last_became_active":"2024-09-13T17:43:36.971235+0000","last_became_peered":"2024-09-13T17:43:36.971235+0000","last_unstale":"2024-09-13T17:43:38.274099+0000","last_undegraded":"2024-09-13T17:43:38.274099+0000","last_fullsized":"2024-09-13T17:43:38.274099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:28.557515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321828+0000","last_change":"2024-09-13T17:43:36.995992+0000","last_active":"2024-09-13T17:43:38.321828+0000","last_peered":"2024-09-13T17:43:38.321828+0000","last_clean":"2024-09-13T17:43:38.321828+0000","last_became_active":"2024-09-13T17:43:36.995839+0000","last_became_peered":"2024-09-13T17:43:36.995839+0000","last_unstale":"2024-09-13T17:43:38.321828+0000","last_undegraded":"2024-09-13T17:43:38.321828+0000","last_fullsized":"2024-09-13T17:43:38.321828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:43.285004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273034+0000","last_change":"2024-09-13T17:43:36.999236+0000","last_active":"2024-09-13T17:43:38.273034+0000","last_peered":"2024-09-13T17:43:38.273034+0000","last_clean":"2024-09-13T17:43:38.273034+0000","last_became_active":"2024-09-13T17:43:36.998617+0000","last_became_peered":"2024-09-13T17:43:36.998617+0000","last_unstale":"2024-09-13T17:43:38.273034+0000","last_undegraded":"2024-09-13T17:43:38.273034+0000","last_fullsized":"2024-09-13T17:43:38.273034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:43.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321605+0000","last_change":"2024-09-13T17:43:36.994453+0000","last_active":"2024-09-13T17:43:38.321605+0000","last_peered":"2024-09-13T17:43:38.321605+0000","last_clean":"2024-09-13T17:43:38.321605+0000","last_became_active":"2024-09-13T17:43:36.993574+0000","last_became_peered":"2024-09-13T17:43:36.993574+0000","last_unstale":"2024-09-13T17:43:38.321605+0000","last_undegraded":"2024-09-13T17:43:38.321605+0000","last_fullsized":"2024-09-13T17:43:38.321605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272616+0000","last_change":"2024-09-13T17:43:36.998865+0000","last_active":"2024-09-13T17:43:38.272616+0000","last_peered":"2024-09-13T17:43:38.272616+0000","last_clean":"2024-09-13T17:43:38.272616+0000","last_became_active":"2024-09-13T17:43:36.998611+0000","last_became_peered":"2024-09-13T17:43:36.998611+0000","last_unstale":"2024-09-13T17:43:38.272616+0000","last_undegraded":"2024-09-13T17:43:38.272616+0000","last_fullsized":"2024-09-13T17:43:38.272616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:31.977360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320600+0000","last_change":"2024-09-13T17:43:36.967204+0000","last_active":"2024-09-13T17:43:38.320600+0000","last_peered":"2024-09-13T17:43:38.320600+0000","last_clean":"2024-09-13T17:43:38.320600+0000","last_became_active":"2024-09-13T17:43:36.966928+0000","last_became_peered":"2024-09-13T17:43:36.966928+0000","last_unstale":"2024-09-13T17:43:38.320600+0000","last_undegraded":"2024-09-13T17:43:38.320600+0000","last_fullsized":"2024-09-13T17:43:38.320600+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:58.179836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950226+0000","last_change":"2024-09-13T17:43:36.989315+0000","last_active":"2024-09-13T17:43:37.950226+0000","last_peered":"2024-09-13T17:43:37.950226+0000","last_clean":"2024-09-13T17:43:37.950226+0000","last_became_active":"2024-09-13T17:43:36.989070+0000","last_became_peered":"2024-09-13T17:43:36.989070+0000","last_unstale":"2024-09-13T17:43:37.950226+0000","last_undegraded":"2024-09-13T17:43:37.950226+0000","last_fullsized":"2024-09-13T17:43:37.950226+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320598+0000","last_change":"2024-09-13T17:43:36.990968+0000","last_active":"2024-09-13T17:43:38.320598+0000","last_peered":"2024-09-13T17:43:38.320598+0000","last_clean":"2024-09-13T17:43:38.320598+0000","last_became_active":"2024-09-13T17:43:36.990688+0000","last_became_peered":"2024-09-13T17:43:36.990688+0000","last_unstale":"2024-09-13T17:43:38.320598+0000","last_undegraded":"2024-09-13T17:43:38.320598+0000","last_fullsized":"2024-09-13T17:43:38.320598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951319+0000","last_change":"2024-09-13T17:43:36.997889+0000","last_active":"2024-09-13T17:43:37.951319+0000","last_peered":"2024-09-13T17:43:37.951319+0000","last_clean":"2024-09-13T17:43:37.951319+0000","last_became_active":"2024-09-13T17:43:36.997671+0000","last_became_peered":"2024-09-13T17:43:36.997671+0000","last_unstale":"2024-09-13T17:43:37.951319+0000","last_undegraded":"2024-09-13T17:43:37.951319+0000","last_fullsized":"2024-09-13T17:43:37.951319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"36'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322431+0000","last_change":"2024-09-13T17:43:36.597308+0000","last_active":"2024-09-13T17:43:38.322431+0000","last_peered":"2024-09-13T17:43:38.322431+0000","last_clean":"2024-09-13T17:43:38.322431+0000","last_became_active":"2024-09-13T17:43:36.596591+0000","last_became_peered":"2024-09-13T17:43:36.596591+0000","last_unstale":"2024-09-13T17:43:38.322431+0000","last_undegraded":"2024-09-13T17:43:38.322431+0000","last_fullsized":"2024-09-13T17:43:38.322431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:03.103037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274269+0000","last_change":"2024-09-13T17:43:36.995868+0000","last_active":"2024-09-13T17:43:38.274269+0000","last_peered":"2024-09-13T17:43:38.274269+0000","last_clean":"2024-09-13T17:43:38.274269+0000","last_became_active":"2024-09-13T17:43:36.995304+0000","last_became_peered":"2024-09-13T17:43:36.995304+0000","last_unstale":"2024-09-13T17:43:38.274269+0000","last_undegraded":"2024-09-13T17:43:38.274269+0000","last_fullsized":"2024-09-13T17:43:38.274269+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:40.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951524+0000","last_change":"2024-09-13T17:43:36.593051+0000","last_active":"2024-09-13T17:43:37.951524+0000","last_peered":"2024-09-13T17:43:37.951524+0000","last_clean":"2024-09-13T17:43:37.951524+0000","last_became_active":"2024-09-13T17:43:36.592210+0000","last_became_peered":"2024-09-13T17:43:36.592210+0000","last_unstale":"2024-09-13T17:43:37.951524+0000","last_undegraded":"2024-09-13T17:43:37.951524+0000","last_fullsized":"2024-09-13T17:43:37.951524+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:07.519036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321670+0000","last_change":"2024-09-13T17:43:36.994621+0000","last_active":"2024-09-13T17:43:38.321670+0000","last_peered":"2024-09-13T17:43:38.321670+0000","last_clean":"2024-09-13T17:43:38.321670+0000","last_became_active":"2024-09-13T17:43:36.993891+0000","last_became_peered":"2024-09-13T17:43:36.993891+0000","last_unstale":"2024-09-13T17:43:38.321670+0000","last_undegraded":"2024-09-13T17:43:38.321670+0000","last_fullsized":"2024-09-13T17:43:38.321670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951704+0000","last_change":"2024-09-13T17:43:36.607525+0000","last_active":"2024-09-13T17:43:37.951704+0000","last_peered":"2024-09-13T17:43:37.951704+0000","last_clean":"2024-09-13T17:43:37.951704+0000","last_became_active":"2024-09-13T17:43:36.606688+0000","last_became_peered":"2024-09-13T17:43:36.606688+0000","last_unstale":"2024-09-13T17:43:37.951704+0000","last_undegraded":"2024-09-13T17:43:37.951704+0000","last_fullsized":"2024-09-13T17:43:37.951704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:34.433256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951210+0000","last_change":"2024-09-13T17:43:36.987727+0000","last_active":"2024-09-13T17:43:37.951210+0000","last_peered":"2024-09-13T17:43:37.951210+0000","last_clean":"2024-09-13T17:43:37.951210+0000","last_became_active":"2024-09-13T17:43:36.987360+0000","last_became_peered":"2024-09-13T17:43:36.987360+0000","last_unstale":"2024-09-13T17:43:37.951210+0000","last_undegraded":"2024-09-13T17:43:37.951210+0000","last_fullsized":"2024-09-13T17:43:37.951210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951049+0000","last_change":"2024-09-13T17:43:36.604117+0000","last_active":"2024-09-13T17:43:37.951049+0000","last_peered":"2024-09-13T17:43:37.951049+0000","last_clean":"2024-09-13T17:43:37.951049+0000","last_became_active":"2024-09-13T17:43:36.603754+0000","last_became_peered":"2024-09-13T17:43:36.603754+0000","last_unstale":"2024-09-13T17:43:37.951049+0000","last_undegraded":"2024-09-13T17:43:37.951049+0000","last_fullsized":"2024-09-13T17:43:37.951049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:48.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321117+0000","last_change":"2024-09-13T17:43:36.976896+0000","last_active":"2024-09-13T17:43:38.321117+0000","last_peered":"2024-09-13T17:43:38.321117+0000","last_clean":"2024-09-13T17:43:38.321117+0000","last_became_active":"2024-09-13T17:43:36.974032+0000","last_became_peered":"2024-09-13T17:43:36.974032+0000","last_unstale":"2024-09-13T17:43:38.321117+0000","last_undegraded":"2024-09-13T17:43:38.321117+0000","last_fullsized":"2024-09-13T17:43:38.321117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:52.403736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273233+0000","last_change":"2024-09-13T17:43:36.601888+0000","last_active":"2024-09-13T17:43:38.273233+0000","last_peered":"2024-09-13T17:43:38.273233+0000","last_clean":"2024-09-13T17:43:38.273233+0000","last_became_active":"2024-09-13T17:43:36.601588+0000","last_became_peered":"2024-09-13T17:43:36.601588+0000","last_unstale":"2024-09-13T17:43:38.273233+0000","last_undegraded":"2024-09-13T17:43:38.273233+0000","last_fullsized":"2024-09-13T17:43:38.273233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:05.159220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272580+0000","last_change":"2024-09-13T17:43:36.976787+0000","last_active":"2024-09-13T17:43:38.272580+0000","last_peered":"2024-09-13T17:43:38.272580+0000","last_clean":"2024-09-13T17:43:38.272580+0000","last_became_active":"2024-09-13T17:43:36.976205+0000","last_became_peered":"2024-09-13T17:43:36.976205+0000","last_unstale":"2024-09-13T17:43:38.272580+0000","last_undegraded":"2024-09-13T17:43:38.272580+0000","last_fullsized":"2024-09-13T17:43:38.272580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:09.035544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272638+0000","last_change":"2024-09-13T17:43:36.612531+0000","last_active":"2024-09-13T17:43:38.272638+0000","last_peered":"2024-09-13T17:43:38.272638+0000","last_clean":"2024-09-13T17:43:38.272638+0000","last_became_active":"2024-09-13T17:43:36.610289+0000","last_became_peered":"2024-09-13T17:43:36.610289+0000","last_unstale":"2024-09-13T17:43:38.272638+0000","last_undegraded":"2024-09-13T17:43:38.272638+0000","last_fullsized":"2024-09-13T17:43:38.272638+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:09.724844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271583+0000","last_change":"2024-09-13T17:43:36.991160+0000","last_active":"2024-09-13T17:43:38.271583+0000","last_peered":"2024-09-13T17:43:38.271583+0000","last_clean":"2024-09-13T17:43:38.271583+0000","last_became_active":"2024-09-13T17:43:36.990271+0000","last_became_peered":"2024-09-13T17:43:36.990271+0000","last_unstale":"2024-09-13T17:43:38.271583+0000","last_undegraded":"2024-09-13T17:43:38.271583+0000","last_fullsized":"2024-09-13T17:43:38.271583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:03.041558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272381+0000","last_change":"2024-09-13T17:43:36.612265+0000","last_active":"2024-09-13T17:43:38.272381+0000","last_peered":"2024-09-13T17:43:38.272381+0000","last_clean":"2024-09-13T17:43:38.272381+0000","last_became_active":"2024-09-13T17:43:36.609463+0000","last_became_peered":"2024-09-13T17:43:36.609463+0000","last_unstale":"2024-09-13T17:43:38.272381+0000","last_undegraded":"2024-09-13T17:43:38.272381+0000","last_fullsized":"2024-09-13T17:43:38.272381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:51.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950545+0000","last_change":"2024-09-13T17:43:36.988557+0000","last_active":"2024-09-13T17:43:37.950545+0000","last_peered":"2024-09-13T17:43:37.950545+0000","last_clean":"2024-09-13T17:43:37.950545+0000","last_became_active":"2024-09-13T17:43:36.985705+0000","last_became_peered":"2024-09-13T17:43:36.985705+0000","last_unstale":"2024-09-13T17:43:37.950545+0000","last_undegraded":"2024-09-13T17:43:37.950545+0000","last_fullsized":"2024-09-13T17:43:37.950545+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271669+0000","last_change":"2024-09-13T17:43:36.614029+0000","last_active":"2024-09-13T17:43:38.271669+0000","last_peered":"2024-09-13T17:43:38.271669+0000","last_clean":"2024-09-13T17:43:38.271669+0000","last_became_active":"2024-09-13T17:43:36.612938+0000","last_became_peered":"2024-09-13T17:43:36.612938+0000","last_unstale":"2024-09-13T17:43:38.271669+0000","last_undegraded":"2024-09-13T17:43:38.271669+0000","last_fullsized":"2024-09-13T17:43:38.271669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:57.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951416+0000","last_change":"2024-09-13T17:43:36.994355+0000","last_active":"2024-09-13T17:43:37.951416+0000","last_peered":"2024-09-13T17:43:37.951416+0000","last_clean":"2024-09-13T17:43:37.951416+0000","last_became_active":"2024-09-13T17:43:36.988868+0000","last_became_peered":"2024-09-13T17:43:36.988868+0000","last_unstale":"2024-09-13T17:43:37.951416+0000","last_undegraded":"2024-09-13T17:43:37.951416+0000","last_fullsized":"2024-09-13T17:43:37.951416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951862+0000","last_change":"2024-09-13T17:43:36.606733+0000","last_active":"2024-09-13T17:43:37.951862+0000","last_peered":"2024-09-13T17:43:37.951862+0000","last_clean":"2024-09-13T17:43:37.951862+0000","last_became_active":"2024-09-13T17:43:36.604421+0000","last_became_peered":"2024-09-13T17:43:36.604421+0000","last_unstale":"2024-09-13T17:43:37.951862+0000","last_undegraded":"2024-09-13T17:43:37.951862+0000","last_fullsized":"2024-09-13T17:43:37.951862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:21.923719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951610+0000","last_change":"2024-09-13T17:43:36.989522+0000","last_active":"2024-09-13T17:43:37.951610+0000","last_peered":"2024-09-13T17:43:37.951610+0000","last_clean":"2024-09-13T17:43:37.951610+0000","last_became_active":"2024-09-13T17:43:36.987659+0000","last_became_peered":"2024-09-13T17:43:36.987659+0000","last_unstale":"2024-09-13T17:43:37.951610+0000","last_undegraded":"2024-09-13T17:43:37.951610+0000","last_fullsized":"2024-09-13T17:43:37.951610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274039+0000","last_change":"2024-09-13T17:43:36.610299+0000","last_active":"2024-09-13T17:43:38.274039+0000","last_peered":"2024-09-13T17:43:38.274039+0000","last_clean":"2024-09-13T17:43:38.274039+0000","last_became_active":"2024-09-13T17:43:36.609995+0000","last_became_peered":"2024-09-13T17:43:36.609995+0000","last_unstale":"2024-09-13T17:43:38.274039+0000","last_undegraded":"2024-09-13T17:43:38.274039+0000","last_fullsized":"2024-09-13T17:43:38.274039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:06.509485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950972+0000","last_change":"2024-09-13T17:43:36.988822+0000","last_active":"2024-09-13T17:43:37.950972+0000","last_peered":"2024-09-13T17:43:37.950972+0000","last_clean":"2024-09-13T17:43:37.950972+0000","last_became_active":"2024-09-13T17:43:36.987189+0000","last_became_peered":"2024-09-13T17:43:36.987189+0000","last_unstale":"2024-09-13T17:43:37.950972+0000","last_undegraded":"2024-09-13T17:43:37.950972+0000","last_fullsized":"2024-09-13T17:43:37.950972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321535+0000","last_change":"2024-09-13T17:43:36.604426+0000","last_active":"2024-09-13T17:43:38.321535+0000","last_peered":"2024-09-13T17:43:38.321535+0000","last_clean":"2024-09-13T17:43:38.321535+0000","last_became_active":"2024-09-13T17:43:36.603659+0000","last_became_peered":"2024-09-13T17:43:36.603659+0000","last_unstale":"2024-09-13T17:43:38.321535+0000","last_undegraded":"2024-09-13T17:43:38.321535+0000","last_fullsized":"2024-09-13T17:43:38.321535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:27.409650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321377+0000","last_change":"2024-09-13T17:43:36.991166+0000","last_active":"2024-09-13T17:43:38.321377+0000","last_peered":"2024-09-13T17:43:38.321377+0000","last_clean":"2024-09-13T17:43:38.321377+0000","last_became_active":"2024-09-13T17:43:36.990864+0000","last_became_peered":"2024-09-13T17:43:36.990864+0000","last_unstale":"2024-09-13T17:43:38.321377+0000","last_undegraded":"2024-09-13T17:43:38.321377+0000","last_fullsized":"2024-09-13T17:43:38.321377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322085+0000","last_change":"2024-09-13T17:43:36.603712+0000","last_active":"2024-09-13T17:43:38.322085+0000","last_peered":"2024-09-13T17:43:38.322085+0000","last_clean":"2024-09-13T17:43:38.322085+0000","last_became_active":"2024-09-13T17:43:36.603454+0000","last_became_peered":"2024-09-13T17:43:36.603454+0000","last_unstale":"2024-09-13T17:43:38.322085+0000","last_undegraded":"2024-09-13T17:43:38.322085+0000","last_fullsized":"2024-09-13T17:43:38.322085+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:01.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950478+0000","last_change":"2024-09-13T17:43:36.988758+0000","last_active":"2024-09-13T17:43:37.950478+0000","last_peered":"2024-09-13T17:43:37.950478+0000","last_clean":"2024-09-13T17:43:37.950478+0000","last_became_active":"2024-09-13T17:43:36.987064+0000","last_became_peered":"2024-09-13T17:43:36.987064+0000","last_unstale":"2024-09-13T17:43:37.950478+0000","last_undegraded":"2024-09-13T17:43:37.950478+0000","last_fullsized":"2024-09-13T17:43:37.950478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321328+0000","last_change":"2024-09-13T17:43:36.590347+0000","last_active":"2024-09-13T17:43:38.321328+0000","last_peered":"2024-09-13T17:43:38.321328+0000","last_clean":"2024-09-13T17:43:38.321328+0000","last_became_active":"2024-09-13T17:43:36.589982+0000","last_became_peered":"2024-09-13T17:43:36.589982+0000","last_unstale":"2024-09-13T17:43:38.321328+0000","last_undegraded":"2024-09-13T17:43:38.321328+0000","last_fullsized":"2024-09-13T17:43:38.321328+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:03.593342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273401+0000","last_change":"2024-09-13T17:43:36.976611+0000","last_active":"2024-09-13T17:43:38.273401+0000","last_peered":"2024-09-13T17:43:38.273401+0000","last_clean":"2024-09-13T17:43:38.273401+0000","last_became_active":"2024-09-13T17:43:36.972871+0000","last_became_peered":"2024-09-13T17:43:36.972871+0000","last_unstale":"2024-09-13T17:43:38.273401+0000","last_undegraded":"2024-09-13T17:43:38.273401+0000","last_fullsized":"2024-09-13T17:43:38.273401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:02.818469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272719+0000","last_change":"2024-09-13T17:43:36.602194+0000","last_active":"2024-09-13T17:43:38.272719+0000","last_peered":"2024-09-13T17:43:38.272719+0000","last_clean":"2024-09-13T17:43:38.272719+0000","last_became_active":"2024-09-13T17:43:36.601939+0000","last_became_peered":"2024-09-13T17:43:36.601939+0000","last_unstale":"2024-09-13T17:43:38.272719+0000","last_undegraded":"2024-09-13T17:43:38.272719+0000","last_fullsized":"2024-09-13T17:43:38.272719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:32.706151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950799+0000","last_change":"2024-09-13T17:43:36.980350+0000","last_active":"2024-09-13T17:43:37.950799+0000","last_peered":"2024-09-13T17:43:37.950799+0000","last_clean":"2024-09-13T17:43:37.950799+0000","last_became_active":"2024-09-13T17:43:36.977778+0000","last_became_peered":"2024-09-13T17:43:36.977778+0000","last_unstale":"2024-09-13T17:43:37.950799+0000","last_undegraded":"2024-09-13T17:43:37.950799+0000","last_fullsized":"2024-09-13T17:43:37.950799+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320658+0000","last_change":"2024-09-13T17:43:36.597400+0000","last_active":"2024-09-13T17:43:38.320658+0000","last_peered":"2024-09-13T17:43:38.320658+0000","last_clean":"2024-09-13T17:43:38.320658+0000","last_became_active":"2024-09-13T17:43:36.596736+0000","last_became_peered":"2024-09-13T17:43:36.596736+0000","last_unstale":"2024-09-13T17:43:38.320658+0000","last_undegraded":"2024-09-13T17:43:38.320658+0000","last_fullsized":"2024-09-13T17:43:38.320658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:21.792293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320430+0000","last_change":"2024-09-13T17:43:36.967452+0000","last_active":"2024-09-13T17:43:38.320430+0000","last_peered":"2024-09-13T17:43:38.320430+0000","last_clean":"2024-09-13T17:43:38.320430+0000","last_became_active":"2024-09-13T17:43:36.966901+0000","last_became_peered":"2024-09-13T17:43:36.966901+0000","last_unstale":"2024-09-13T17:43:38.320430+0000","last_undegraded":"2024-09-13T17:43:38.320430+0000","last_fullsized":"2024-09-13T17:43:38.320430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"36'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271906+0000","last_change":"2024-09-13T17:43:36.607932+0000","last_active":"2024-09-13T17:43:38.271906+0000","last_peered":"2024-09-13T17:43:38.271906+0000","last_clean":"2024-09-13T17:43:38.271906+0000","last_became_active":"2024-09-13T17:43:36.607651+0000","last_became_peered":"2024-09-13T17:43:36.607651+0000","last_unstale":"2024-09-13T17:43:38.271906+0000","last_undegraded":"2024-09-13T17:43:38.271906+0000","last_fullsized":"2024-09-13T17:43:38.271906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:09.002010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321981+0000","last_change":"2024-09-13T17:43:36.996236+0000","last_active":"2024-09-13T17:43:38.321981+0000","last_peered":"2024-09-13T17:43:38.321981+0000","last_clean":"2024-09-13T17:43:38.321981+0000","last_became_active":"2024-09-13T17:43:36.995955+0000","last_became_peered":"2024-09-13T17:43:36.995955+0000","last_unstale":"2024-09-13T17:43:38.321981+0000","last_undegraded":"2024-09-13T17:43:38.321981+0000","last_fullsized":"2024-09-13T17:43:38.321981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:23.137553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322377+0000","last_change":"2024-09-13T17:43:36.602229+0000","last_active":"2024-09-13T17:43:38.322377+0000","last_peered":"2024-09-13T17:43:38.322377+0000","last_clean":"2024-09-13T17:43:38.322377+0000","last_became_active":"2024-09-13T17:43:36.601122+0000","last_became_peered":"2024-09-13T17:43:36.601122+0000","last_unstale":"2024-09-13T17:43:38.322377+0000","last_undegraded":"2024-09-13T17:43:38.322377+0000","last_fullsized":"2024-09-13T17:43:38.322377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:25.802228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274309+0000","last_change":"2024-09-13T17:43:36.996063+0000","last_active":"2024-09-13T17:43:38.274309+0000","last_peered":"2024-09-13T17:43:38.274309+0000","last_clean":"2024-09-13T17:43:38.274309+0000","last_became_active":"2024-09-13T17:43:36.995621+0000","last_became_peered":"2024-09-13T17:43:36.995621+0000","last_unstale":"2024-09-13T17:43:38.274309+0000","last_undegraded":"2024-09-13T17:43:38.274309+0000","last_fullsized":"2024-09-13T17:43:38.274309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:12.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951614+0000","last_change":"2024-09-13T17:43:36.594362+0000","last_active":"2024-09-13T17:43:37.951614+0000","last_peered":"2024-09-13T17:43:37.951614+0000","last_clean":"2024-09-13T17:43:37.951614+0000","last_became_active":"2024-09-13T17:43:36.593113+0000","last_became_peered":"2024-09-13T17:43:36.593113+0000","last_unstale":"2024-09-13T17:43:37.951614+0000","last_undegraded":"2024-09-13T17:43:37.951614+0000","last_fullsized":"2024-09-13T17:43:37.951614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:01.736566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321882+0000","last_change":"2024-09-13T17:43:36.994777+0000","last_active":"2024-09-13T17:43:38.321882+0000","last_peered":"2024-09-13T17:43:38.321882+0000","last_clean":"2024-09-13T17:43:38.321882+0000","last_became_active":"2024-09-13T17:43:36.994213+0000","last_became_peered":"2024-09-13T17:43:36.994213+0000","last_unstale":"2024-09-13T17:43:38.321882+0000","last_undegraded":"2024-09-13T17:43:38.321882+0000","last_fullsized":"2024-09-13T17:43:38.321882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951755+0000","last_change":"2024-09-13T17:43:36.607688+0000","last_active":"2024-09-13T17:43:37.951755+0000","last_peered":"2024-09-13T17:43:37.951755+0000","last_clean":"2024-09-13T17:43:37.951755+0000","last_became_active":"2024-09-13T17:43:36.607290+0000","last_became_peered":"2024-09-13T17:43:36.607290+0000","last_unstale":"2024-09-13T17:43:37.951755+0000","last_undegraded":"2024-09-13T17:43:37.951755+0000","last_fullsized":"2024-09-13T17:43:37.951755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:24.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951500+0000","last_change":"2024-09-13T17:43:36.987379+0000","last_active":"2024-09-13T17:43:37.951500+0000","last_peered":"2024-09-13T17:43:37.951500+0000","last_clean":"2024-09-13T17:43:37.951500+0000","last_became_active":"2024-09-13T17:43:36.986990+0000","last_became_peered":"2024-09-13T17:43:36.986990+0000","last_unstale":"2024-09-13T17:43:37.951500+0000","last_undegraded":"2024-09-13T17:43:37.951500+0000","last_fullsized":"2024-09-13T17:43:37.951500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951146+0000","last_change":"2024-09-13T17:43:36.596481+0000","last_active":"2024-09-13T17:43:37.951146+0000","last_peered":"2024-09-13T17:43:37.951146+0000","last_clean":"2024-09-13T17:43:37.951146+0000","last_became_active":"2024-09-13T17:43:36.594330+0000","last_became_peered":"2024-09-13T17:43:36.594330+0000","last_unstale":"2024-09-13T17:43:37.951146+0000","last_undegraded":"2024-09-13T17:43:37.951146+0000","last_fullsized":"2024-09-13T17:43:37.951146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:52.485090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321058+0000","last_change":"2024-09-13T17:43:36.977463+0000","last_active":"2024-09-13T17:43:38.321058+0000","last_peered":"2024-09-13T17:43:38.321058+0000","last_clean":"2024-09-13T17:43:38.321058+0000","last_became_active":"2024-09-13T17:43:36.977207+0000","last_became_peered":"2024-09-13T17:43:36.977207+0000","last_unstale":"2024-09-13T17:43:38.321058+0000","last_undegraded":"2024-09-13T17:43:38.321058+0000","last_fullsized":"2024-09-13T17:43:38.321058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:02:51.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951408+0000","last_change":"2024-09-13T17:43:36.607315+0000","last_active":"2024-09-13T17:43:37.951408+0000","last_peered":"2024-09-13T17:43:37.951408+0000","last_clean":"2024-09-13T17:43:37.951408+0000","last_became_active":"2024-09-13T17:43:36.606824+0000","last_became_peered":"2024-09-13T17:43:36.606824+0000","last_unstale":"2024-09-13T17:43:37.951408+0000","last_undegraded":"2024-09-13T17:43:37.951408+0000","last_fullsized":"2024-09-13T17:43:37.951408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.362407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950605+0000","last_change":"2024-09-13T17:43:36.991670+0000","last_active":"2024-09-13T17:43:37.950605+0000","last_peered":"2024-09-13T17:43:37.950605+0000","last_clean":"2024-09-13T17:43:37.950605+0000","last_became_active":"2024-09-13T17:43:36.987763+0000","last_became_peered":"2024-09-13T17:43:36.987763+0000","last_unstale":"2024-09-13T17:43:37.950605+0000","last_undegraded":"2024-09-13T17:43:37.950605+0000","last_fullsized":"2024-09-13T17:43:37.950605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272743+0000","last_change":"2024-09-13T17:43:36.612138+0000","last_active":"2024-09-13T17:43:38.272743+0000","last_peered":"2024-09-13T17:43:38.272743+0000","last_clean":"2024-09-13T17:43:38.272743+0000","last_became_active":"2024-09-13T17:43:36.609873+0000","last_became_peered":"2024-09-13T17:43:36.609873+0000","last_unstale":"2024-09-13T17:43:38.272743+0000","last_undegraded":"2024-09-13T17:43:38.272743+0000","last_fullsized":"2024-09-13T17:43:38.272743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:49.578767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320804+0000","last_change":"2024-09-13T17:43:36.990056+0000","last_active":"2024-09-13T17:43:38.320804+0000","last_peered":"2024-09-13T17:43:38.320804+0000","last_clean":"2024-09-13T17:43:38.320804+0000","last_became_active":"2024-09-13T17:43:36.989640+0000","last_became_peered":"2024-09-13T17:43:36.989640+0000","last_unstale":"2024-09-13T17:43:38.320804+0000","last_undegraded":"2024-09-13T17:43:38.320804+0000","last_fullsized":"2024-09-13T17:43:38.320804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272372+0000","last_change":"2024-09-13T17:43:36.612085+0000","last_active":"2024-09-13T17:43:38.272372+0000","last_peered":"2024-09-13T17:43:38.272372+0000","last_clean":"2024-09-13T17:43:38.272372+0000","last_became_active":"2024-09-13T17:43:36.609135+0000","last_became_peered":"2024-09-13T17:43:36.609135+0000","last_unstale":"2024-09-13T17:43:38.272372+0000","last_undegraded":"2024-09-13T17:43:38.272372+0000","last_fullsized":"2024-09-13T17:43:38.272372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:22.346224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271989+0000","last_change":"2024-09-13T17:43:36.999734+0000","last_active":"2024-09-13T17:43:38.271989+0000","last_peered":"2024-09-13T17:43:38.271989+0000","last_clean":"2024-09-13T17:43:38.271989+0000","last_became_active":"2024-09-13T17:43:36.999416+0000","last_became_peered":"2024-09-13T17:43:36.999416+0000","last_unstale":"2024-09-13T17:43:38.271989+0000","last_undegraded":"2024-09-13T17:43:38.271989+0000","last_fullsized":"2024-09-13T17:43:38.271989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:11.465274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320850+0000","last_change":"2024-09-13T17:43:36.598625+0000","last_active":"2024-09-13T17:43:38.320850+0000","last_peered":"2024-09-13T17:43:38.320850+0000","last_clean":"2024-09-13T17:43:38.320850+0000","last_became_active":"2024-09-13T17:43:36.598229+0000","last_became_peered":"2024-09-13T17:43:36.598229+0000","last_unstale":"2024-09-13T17:43:38.320850+0000","last_undegraded":"2024-09-13T17:43:38.320850+0000","last_fullsized":"2024-09-13T17:43:38.320850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:54.846315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951787+0000","last_change":"2024-09-13T17:43:36.988413+0000","last_active":"2024-09-13T17:43:37.951787+0000","last_peered":"2024-09-13T17:43:37.951787+0000","last_clean":"2024-09-13T17:43:37.951787+0000","last_became_active":"2024-09-13T17:43:36.987084+0000","last_became_peered":"2024-09-13T17:43:36.987084+0000","last_unstale":"2024-09-13T17:43:37.951787+0000","last_undegraded":"2024-09-13T17:43:37.951787+0000","last_fullsized":"2024-09-13T17:43:37.951787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951602+0000","last_change":"2024-09-13T17:43:36.596762+0000","last_active":"2024-09-13T17:43:37.951602+0000","last_peered":"2024-09-13T17:43:37.951602+0000","last_clean":"2024-09-13T17:43:37.951602+0000","last_became_active":"2024-09-13T17:43:36.594514+0000","last_became_peered":"2024-09-13T17:43:36.594514+0000","last_unstale":"2024-09-13T17:43:37.951602+0000","last_undegraded":"2024-09-13T17:43:37.951602+0000","last_fullsized":"2024-09-13T17:43:37.951602+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:30:50.353395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273094+0000","last_change":"2024-09-13T17:43:36.999031+0000","last_active":"2024-09-13T17:43:38.273094+0000","last_peered":"2024-09-13T17:43:38.273094+0000","last_clean":"2024-09-13T17:43:38.273094+0000","last_became_active":"2024-09-13T17:43:36.998251+0000","last_became_peered":"2024-09-13T17:43:36.998251+0000","last_unstale":"2024-09-13T17:43:38.273094+0000","last_undegraded":"2024-09-13T17:43:38.273094+0000","last_fullsized":"2024-09-13T17:43:38.273094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:32.921571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273585+0000","last_change":"2024-09-13T17:43:36.612328+0000","last_active":"2024-09-13T17:43:38.273585+0000","last_peered":"2024-09-13T17:43:38.273585+0000","last_clean":"2024-09-13T17:43:38.273585+0000","last_became_active":"2024-09-13T17:43:36.610325+0000","last_became_peered":"2024-09-13T17:43:36.610325+0000","last_unstale":"2024-09-13T17:43:38.273585+0000","last_undegraded":"2024-09-13T17:43:38.273585+0000","last_fullsized":"2024-09-13T17:43:38.273585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:31.356170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321382+0000","last_change":"2024-09-13T17:43:36.973857+0000","last_active":"2024-09-13T17:43:38.321382+0000","last_peered":"2024-09-13T17:43:38.321382+0000","last_clean":"2024-09-13T17:43:38.321382+0000","last_became_active":"2024-09-13T17:43:36.973595+0000","last_became_peered":"2024-09-13T17:43:36.973595+0000","last_unstale":"2024-09-13T17:43:38.321382+0000","last_undegraded":"2024-09-13T17:43:38.321382+0000","last_fullsized":"2024-09-13T17:43:38.321382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:06.165795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951161+0000","last_change":"2024-09-13T17:43:36.592618+0000","last_active":"2024-09-13T17:43:37.951161+0000","last_peered":"2024-09-13T17:43:37.951161+0000","last_clean":"2024-09-13T17:43:37.951161+0000","last_became_active":"2024-09-13T17:43:36.592303+0000","last_became_peered":"2024-09-13T17:43:36.592303+0000","last_unstale":"2024-09-13T17:43:37.951161+0000","last_undegraded":"2024-09-13T17:43:37.951161+0000","last_fullsized":"2024-09-13T17:43:37.951161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:16.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321211+0000","last_change":"2024-09-13T17:43:36.994616+0000","last_active":"2024-09-13T17:43:38.321211+0000","last_peered":"2024-09-13T17:43:38.321211+0000","last_clean":"2024-09-13T17:43:38.321211+0000","last_became_active":"2024-09-13T17:43:36.994356+0000","last_became_peered":"2024-09-13T17:43:36.994356+0000","last_unstale":"2024-09-13T17:43:38.321211+0000","last_undegraded":"2024-09-13T17:43:38.321211+0000","last_fullsized":"2024-09-13T17:43:38.321211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273571+0000","last_change":"2024-09-13T17:43:36.601819+0000","last_active":"2024-09-13T17:43:38.273571+0000","last_peered":"2024-09-13T17:43:38.273571+0000","last_clean":"2024-09-13T17:43:38.273571+0000","last_became_active":"2024-09-13T17:43:36.601091+0000","last_became_peered":"2024-09-13T17:43:36.601091+0000","last_unstale":"2024-09-13T17:43:38.273571+0000","last_undegraded":"2024-09-13T17:43:38.273571+0000","last_fullsized":"2024-09-13T17:43:38.273571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:22.130979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272856+0000","last_change":"2024-09-13T17:43:36.976449+0000","last_active":"2024-09-13T17:43:38.272856+0000","last_peered":"2024-09-13T17:43:38.272856+0000","last_clean":"2024-09-13T17:43:38.272856+0000","last_became_active":"2024-09-13T17:43:36.972460+0000","last_became_peered":"2024-09-13T17:43:36.972460+0000","last_unstale":"2024-09-13T17:43:38.272856+0000","last_undegraded":"2024-09-13T17:43:38.272856+0000","last_fullsized":"2024-09-13T17:43:38.272856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:12.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950576+0000","last_change":"2024-09-13T17:43:36.608221+0000","last_active":"2024-09-13T17:43:37.950576+0000","last_peered":"2024-09-13T17:43:37.950576+0000","last_clean":"2024-09-13T17:43:37.950576+0000","last_became_active":"2024-09-13T17:43:36.607779+0000","last_became_peered":"2024-09-13T17:43:36.607779+0000","last_unstale":"2024-09-13T17:43:37.950576+0000","last_undegraded":"2024-09-13T17:43:37.950576+0000","last_fullsized":"2024-09-13T17:43:37.950576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:00.401036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950260+0000","last_change":"2024-09-13T17:43:36.977858+0000","last_active":"2024-09-13T17:43:37.950260+0000","last_peered":"2024-09-13T17:43:37.950260+0000","last_clean":"2024-09-13T17:43:37.950260+0000","last_became_active":"2024-09-13T17:43:36.977282+0000","last_became_peered":"2024-09-13T17:43:36.977282+0000","last_unstale":"2024-09-13T17:43:37.950260+0000","last_undegraded":"2024-09-13T17:43:37.950260+0000","last_fullsized":"2024-09-13T17:43:37.950260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950612+0000","last_change":"2024-09-13T17:43:36.601601+0000","last_active":"2024-09-13T17:43:37.950612+0000","last_peered":"2024-09-13T17:43:37.950612+0000","last_clean":"2024-09-13T17:43:37.950612+0000","last_became_active":"2024-09-13T17:43:36.594738+0000","last_became_peered":"2024-09-13T17:43:36.594738+0000","last_unstale":"2024-09-13T17:43:37.950612+0000","last_undegraded":"2024-09-13T17:43:37.950612+0000","last_fullsized":"2024-09-13T17:43:37.950612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:26.235235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320493+0000","last_change":"2024-09-13T17:43:36.990024+0000","last_active":"2024-09-13T17:43:38.320493+0000","last_peered":"2024-09-13T17:43:38.320493+0000","last_clean":"2024-09-13T17:43:38.320493+0000","last_became_active":"2024-09-13T17:43:36.989774+0000","last_became_peered":"2024-09-13T17:43:36.989774+0000","last_unstale":"2024-09-13T17:43:38.320493+0000","last_undegraded":"2024-09-13T17:43:38.320493+0000","last_fullsized":"2024-09-13T17:43:38.320493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271761+0000","last_change":"2024-09-13T17:43:36.608307+0000","last_active":"2024-09-13T17:43:38.271761+0000","last_peered":"2024-09-13T17:43:38.271761+0000","last_clean":"2024-09-13T17:43:38.271761+0000","last_became_active":"2024-09-13T17:43:36.607846+0000","last_became_peered":"2024-09-13T17:43:36.607846+0000","last_unstale":"2024-09-13T17:43:38.271761+0000","last_undegraded":"2024-09-13T17:43:38.271761+0000","last_fullsized":"2024-09-13T17:43:38.271761+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:34.779723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273934+0000","last_change":"2024-09-13T17:43:36.972148+0000","last_active":"2024-09-13T17:43:38.273934+0000","last_peered":"2024-09-13T17:43:38.273934+0000","last_clean":"2024-09-13T17:43:38.273934+0000","last_became_active":"2024-09-13T17:43:36.971337+0000","last_became_peered":"2024-09-13T17:43:36.971337+0000","last_unstale":"2024-09-13T17:43:38.273934+0000","last_undegraded":"2024-09-13T17:43:38.273934+0000","last_fullsized":"2024-09-13T17:43:38.273934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:15.285058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274593+0000","last_change":"2024-09-13T17:43:36.601639+0000","last_active":"2024-09-13T17:43:38.274593+0000","last_peered":"2024-09-13T17:43:38.274593+0000","last_clean":"2024-09-13T17:43:38.274593+0000","last_became_active":"2024-09-13T17:43:36.601329+0000","last_became_peered":"2024-09-13T17:43:36.601329+0000","last_unstale":"2024-09-13T17:43:38.274593+0000","last_undegraded":"2024-09-13T17:43:38.274593+0000","last_fullsized":"2024-09-13T17:43:38.274593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:43.542629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":239,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T17:43:36.953347+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:43:36.953347+0000","last_peered":"2024-09-13T17:43:36.953347+0000","last_clean":"2024-09-13T17:43:36.953347+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:43:36.953347+0000","last_undegraded":"2024-09-13T17:43:36.953347+0000","last_fullsized":"2024-09-13T17:43:36.953347+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":177,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":177,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738392,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":125763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68100000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738393,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":933888,"data_stored":713965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.504},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":937984,"data_stored":712115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:43:39.908 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:43:40.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.053+0000 7f10e6435640 1 Processor -- start 2024-09-13T17:43:40.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.053+0000 7f10e6435640 1 -- start start 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f10e006f6d0 0x7f10e0134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e006e130 con 0x7f10e009dc20 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e00a6750 con 0x7f10e006f6d0 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f10e006f6d0 0x7f10e0134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f10e006f6d0 0x7f10e0134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50396/0 (socket says 172.21.15.96:50396) 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10df7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f10e009dc20 0x7f10e0134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33864/0 (socket says 172.21.15.96:33864) 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 -- 172.21.15.96:0/294437889 learned_addr learned my addr 172.21.15.96:0/294437889 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 -- 172.21.15.96:0/294437889 >> v1:172.21.15.96:6789/0 conn(0x7f10e009dc20 legacy=0x7f10e0134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 -- 172.21.15.96:0/294437889 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00712e0 con 0x7f10e006f6d0 2024-09-13T17:43:40.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10dffff640 1 --2- 172.21.15.96:0/294437889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e0134ab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f10d400ba90 tx=0x7f10d401eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02bc77633bcc230 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10deffd640 1 -- 172.21.15.96:0/294437889 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10d40a77f0 con 0x7f10e006f6d0 2024-09-13T17:43:40.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10deffd640 1 -- 172.21.15.96:0/294437889 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40a7950 con 0x7f10e006f6d0 2024-09-13T17:43:40.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10deffd640 1 -- 172.21.15.96:0/294437889 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10d40a7c10 con 0x7f10e006f6d0 2024-09-13T17:43:40.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- 172.21.15.96:0/294437889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 msgr2=0x7f10e0134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 --2- 172.21.15.96:0/294437889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e0134ab0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f10d400ba90 tx=0x7f10d401eb00 comp rx=0 tx=0).stop 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- 172.21.15.96:0/294437889 shutdown_connections 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 --2- 172.21.15.96:0/294437889 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e0134ab0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- 172.21.15.96:0/294437889 >> 172.21.15.96:0/294437889 conn(0x7f10e0098a60 msgr2=0x7f10e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- 172.21.15.96:0/294437889 shutdown_connections 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.057+0000 7f10e6435640 1 -- 172.21.15.96:0/294437889 wait complete. 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 Processor -- start 2024-09-13T17:43:40.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 -- start start 2024-09-13T17:43:40.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e00a4b40 con 0x7f10e006f6d0 2024-09-13T17:43:40.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50406/0 (socket says 172.21.15.96:50406) 2024-09-13T17:43:40.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dffff640 1 -- 172.21.15.96:0/1540721732 learned_addr learned my addr 172.21.15.96:0/1540721732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:40.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dffff640 1 -- 172.21.15.96:0/1540721732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d400b770 con 0x7f10e006f6d0 2024-09-13T17:43:40.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dffff640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f10d401c730 tx=0x7f10d401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10d40be070 con 0x7f10e006f6d0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40afa20 con 0x7f10e006f6d0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e009ae60 con 0x7f10e006f6d0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10d40afce0 con 0x7f10e006f6d0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e00a4e50 con 0x7f10e006f6d0 2024-09-13T17:43:40.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f10d40c3410 con 0x7f10e006f6d0 2024-09-13T17:43:40.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e00a4c90 con 0x7f10e006f6d0 2024-09-13T17:43:40.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 0x7f10cc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f10d40e2570 con 0x7f10e006f6d0 2024-09-13T17:43:40.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10df7fe640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 0x7f10cc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.061+0000 7f10df7fe640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 0x7f10cc037a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001600 tx=0x7f10d0007a50 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.065+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10d40ad340 con 0x7f10e006f6d0 2024-09-13T17:43:40.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.209+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 --> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10e0135910 con 0x7f10cc035570 2024-09-13T17:43:40.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.217+0000 7f10dcff9640 1 -- 172.21.15.96:0/1540721732 <== mgr.4630 v2:172.21.15.96:6824/526856805 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249937 (secure 0 0 0) 0x7f10d00070d0 con 0x7f10cc035570 2024-09-13T17:43:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:40.225 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:43:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 msgr2=0x7f10cc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 0x7f10cc037a30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f10d0001600 tx=0x7f10d0007a50 comp rx=0 tx=0).stop 2024-09-13T17:43:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 msgr2=0x7f10e01c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f10d401c730 tx=0x7f10d401d340 comp rx=0 tx=0).stop 2024-09-13T17:43:40.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 shutdown_connections 2024-09-13T17:43:40.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f10cc035570 0x7f10cc037a30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 --2- 172.21.15.96:0/1540721732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e006f6d0 0x7f10e01c82c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 >> 172.21.15.96:0/1540721732 conn(0x7f10e0098a60 msgr2=0x7f10e0132ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 shutdown_connections 2024-09-13T17:43:40.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.221+0000 7f10e6435640 1 -- 172.21.15.96:0/1540721732 wait complete. 2024-09-13T17:43:40.240 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-13T17:43:39.050377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":221,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":91440,"kb_used_data":2176,"kb_used_omap":4,"kb_used_meta":89211,"kb_avail":283024080,"statfs":{"total":289910292480,"available":289816657920,"internally_reserved":0,"allocated":2228224,"data_stored":1551843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":91352403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000825"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273887+0000","last_change":"2024-09-13T17:43:36.972254+0000","last_active":"2024-09-13T17:43:38.273887+0000","last_peered":"2024-09-13T17:43:38.273887+0000","last_clean":"2024-09-13T17:43:38.273887+0000","last_became_active":"2024-09-13T17:43:36.971289+0000","last_became_peered":"2024-09-13T17:43:36.971289+0000","last_unstale":"2024-09-13T17:43:38.273887+0000","last_undegraded":"2024-09-13T17:43:38.273887+0000","last_fullsized":"2024-09-13T17:43:38.273887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:09.542526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321750+0000","last_change":"2024-09-13T17:43:36.974086+0000","last_active":"2024-09-13T17:43:38.321750+0000","last_peered":"2024-09-13T17:43:38.321750+0000","last_clean":"2024-09-13T17:43:38.321750+0000","last_became_active":"2024-09-13T17:43:36.973536+0000","last_became_peered":"2024-09-13T17:43:36.973536+0000","last_unstale":"2024-09-13T17:43:38.321750+0000","last_undegraded":"2024-09-13T17:43:38.321750+0000","last_fullsized":"2024-09-13T17:43:38.321750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:22:24.478487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951430+0000","last_change":"2024-09-13T17:43:36.988922+0000","last_active":"2024-09-13T17:43:37.951430+0000","last_peered":"2024-09-13T17:43:37.951430+0000","last_clean":"2024-09-13T17:43:37.951430+0000","last_became_active":"2024-09-13T17:43:36.988648+0000","last_became_peered":"2024-09-13T17:43:36.988648+0000","last_unstale":"2024-09-13T17:43:37.951430+0000","last_undegraded":"2024-09-13T17:43:37.951430+0000","last_fullsized":"2024-09-13T17:43:37.951430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273031+0000","last_change":"2024-09-13T17:43:36.972979+0000","last_active":"2024-09-13T17:43:38.273031+0000","last_peered":"2024-09-13T17:43:38.273031+0000","last_clean":"2024-09-13T17:43:38.273031+0000","last_became_active":"2024-09-13T17:43:36.972625+0000","last_became_peered":"2024-09-13T17:43:36.972625+0000","last_unstale":"2024-09-13T17:43:38.273031+0000","last_undegraded":"2024-09-13T17:43:38.273031+0000","last_fullsized":"2024-09-13T17:43:38.273031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:52.901570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272355+0000","last_change":"2024-09-13T17:43:36.998726+0000","last_active":"2024-09-13T17:43:38.272355+0000","last_peered":"2024-09-13T17:43:38.272355+0000","last_clean":"2024-09-13T17:43:38.272355+0000","last_became_active":"2024-09-13T17:43:36.998283+0000","last_became_peered":"2024-09-13T17:43:36.998283+0000","last_unstale":"2024-09-13T17:43:38.272355+0000","last_undegraded":"2024-09-13T17:43:38.272355+0000","last_fullsized":"2024-09-13T17:43:38.272355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:03.727034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272105+0000","last_change":"2024-09-13T17:43:36.996807+0000","last_active":"2024-09-13T17:43:38.272105+0000","last_peered":"2024-09-13T17:43:38.272105+0000","last_clean":"2024-09-13T17:43:38.272105+0000","last_became_active":"2024-09-13T17:43:36.996515+0000","last_became_peered":"2024-09-13T17:43:36.996515+0000","last_unstale":"2024-09-13T17:43:38.272105+0000","last_undegraded":"2024-09-13T17:43:38.272105+0000","last_fullsized":"2024-09-13T17:43:38.272105+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:46.786049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272149+0000","last_change":"2024-09-13T17:43:36.999611+0000","last_active":"2024-09-13T17:43:38.272149+0000","last_peered":"2024-09-13T17:43:38.272149+0000","last_clean":"2024-09-13T17:43:38.272149+0000","last_became_active":"2024-09-13T17:43:36.998996+0000","last_became_peered":"2024-09-13T17:43:36.998996+0000","last_unstale":"2024-09-13T17:43:38.272149+0000","last_undegraded":"2024-09-13T17:43:38.272149+0000","last_fullsized":"2024-09-13T17:43:38.272149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:40.880802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271469+0000","last_change":"2024-09-13T17:43:36.997792+0000","last_active":"2024-09-13T17:43:38.271469+0000","last_peered":"2024-09-13T17:43:38.271469+0000","last_clean":"2024-09-13T17:43:38.271469+0000","last_became_active":"2024-09-13T17:43:36.997361+0000","last_became_peered":"2024-09-13T17:43:36.997361+0000","last_unstale":"2024-09-13T17:43:38.271469+0000","last_undegraded":"2024-09-13T17:43:38.271469+0000","last_fullsized":"2024-09-13T17:43:38.271469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:40.506501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274050+0000","last_change":"2024-09-13T17:43:36.998418+0000","last_active":"2024-09-13T17:43:38.274050+0000","last_peered":"2024-09-13T17:43:38.274050+0000","last_clean":"2024-09-13T17:43:38.274050+0000","last_became_active":"2024-09-13T17:43:36.998145+0000","last_became_peered":"2024-09-13T17:43:36.998145+0000","last_unstale":"2024-09-13T17:43:38.274050+0000","last_undegraded":"2024-09-13T17:43:38.274050+0000","last_fullsized":"2024-09-13T17:43:38.274050+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:12.103252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273758+0000","last_change":"2024-09-13T17:43:36.972143+0000","last_active":"2024-09-13T17:43:38.273758+0000","last_peered":"2024-09-13T17:43:38.273758+0000","last_clean":"2024-09-13T17:43:38.273758+0000","last_became_active":"2024-09-13T17:43:36.971262+0000","last_became_peered":"2024-09-13T17:43:36.971262+0000","last_unstale":"2024-09-13T17:43:38.273758+0000","last_undegraded":"2024-09-13T17:43:38.273758+0000","last_fullsized":"2024-09-13T17:43:38.273758+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:24.359313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273332+0000","last_change":"2024-09-13T17:43:36.999259+0000","last_active":"2024-09-13T17:43:38.273332+0000","last_peered":"2024-09-13T17:43:38.273332+0000","last_clean":"2024-09-13T17:43:38.273332+0000","last_became_active":"2024-09-13T17:43:36.998970+0000","last_became_peered":"2024-09-13T17:43:36.998970+0000","last_unstale":"2024-09-13T17:43:38.273332+0000","last_undegraded":"2024-09-13T17:43:38.273332+0000","last_fullsized":"2024-09-13T17:43:38.273332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:26.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321316+0000","last_change":"2024-09-13T17:43:36.970077+0000","last_active":"2024-09-13T17:43:38.321316+0000","last_peered":"2024-09-13T17:43:38.321316+0000","last_clean":"2024-09-13T17:43:38.321316+0000","last_became_active":"2024-09-13T17:43:36.969731+0000","last_became_peered":"2024-09-13T17:43:36.969731+0000","last_unstale":"2024-09-13T17:43:38.321316+0000","last_undegraded":"2024-09-13T17:43:38.321316+0000","last_fullsized":"2024-09-13T17:43:38.321316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321054+0000","last_change":"2024-09-13T17:43:36.989784+0000","last_active":"2024-09-13T17:43:38.321054+0000","last_peered":"2024-09-13T17:43:38.321054+0000","last_clean":"2024-09-13T17:43:38.321054+0000","last_became_active":"2024-09-13T17:43:36.989513+0000","last_became_peered":"2024-09-13T17:43:36.989513+0000","last_unstale":"2024-09-13T17:43:38.321054+0000","last_undegraded":"2024-09-13T17:43:38.321054+0000","last_fullsized":"2024-09-13T17:43:38.321054+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272297+0000","last_change":"2024-09-13T17:43:36.998495+0000","last_active":"2024-09-13T17:43:38.272297+0000","last_peered":"2024-09-13T17:43:38.272297+0000","last_clean":"2024-09-13T17:43:38.272297+0000","last_became_active":"2024-09-13T17:43:36.998203+0000","last_became_peered":"2024-09-13T17:43:36.998203+0000","last_unstale":"2024-09-13T17:43:38.272297+0000","last_undegraded":"2024-09-13T17:43:38.272297+0000","last_fullsized":"2024-09-13T17:43:38.272297+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:39.769029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320313+0000","last_change":"2024-09-13T17:43:36.989971+0000","last_active":"2024-09-13T17:43:38.320313+0000","last_peered":"2024-09-13T17:43:38.320313+0000","last_clean":"2024-09-13T17:43:38.320313+0000","last_became_active":"2024-09-13T17:43:36.989598+0000","last_became_peered":"2024-09-13T17:43:36.989598+0000","last_unstale":"2024-09-13T17:43:38.320313+0000","last_undegraded":"2024-09-13T17:43:38.320313+0000","last_fullsized":"2024-09-13T17:43:38.320313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271593+0000","last_change":"2024-09-13T17:43:36.997635+0000","last_active":"2024-09-13T17:43:38.271593+0000","last_peered":"2024-09-13T17:43:38.271593+0000","last_clean":"2024-09-13T17:43:38.271593+0000","last_became_active":"2024-09-13T17:43:36.997014+0000","last_became_peered":"2024-09-13T17:43:36.997014+0000","last_unstale":"2024-09-13T17:43:38.271593+0000","last_undegraded":"2024-09-13T17:43:38.271593+0000","last_fullsized":"2024-09-13T17:43:38.271593+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:36.971585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322018+0000","last_change":"2024-09-13T17:43:36.976795+0000","last_active":"2024-09-13T17:43:38.322018+0000","last_peered":"2024-09-13T17:43:38.322018+0000","last_clean":"2024-09-13T17:43:38.322018+0000","last_became_active":"2024-09-13T17:43:36.974036+0000","last_became_peered":"2024-09-13T17:43:36.974036+0000","last_unstale":"2024-09-13T17:43:38.322018+0000","last_undegraded":"2024-09-13T17:43:38.322018+0000","last_fullsized":"2024-09-13T17:43:38.322018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951220+0000","last_change":"2024-09-13T17:43:36.991727+0000","last_active":"2024-09-13T17:43:37.951220+0000","last_peered":"2024-09-13T17:43:37.951220+0000","last_clean":"2024-09-13T17:43:37.951220+0000","last_became_active":"2024-09-13T17:43:36.988498+0000","last_became_peered":"2024-09-13T17:43:36.988498+0000","last_unstale":"2024-09-13T17:43:37.951220+0000","last_undegraded":"2024-09-13T17:43:37.951220+0000","last_fullsized":"2024-09-13T17:43:37.951220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273424+0000","last_change":"2024-09-13T17:43:36.996728+0000","last_active":"2024-09-13T17:43:38.273424+0000","last_peered":"2024-09-13T17:43:38.273424+0000","last_clean":"2024-09-13T17:43:38.273424+0000","last_became_active":"2024-09-13T17:43:36.996468+0000","last_became_peered":"2024-09-13T17:43:36.996468+0000","last_unstale":"2024-09-13T17:43:38.273424+0000","last_undegraded":"2024-09-13T17:43:38.273424+0000","last_fullsized":"2024-09-13T17:43:38.273424+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:56.456764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321323+0000","last_change":"2024-09-13T17:43:36.976924+0000","last_active":"2024-09-13T17:43:38.321323+0000","last_peered":"2024-09-13T17:43:38.321323+0000","last_clean":"2024-09-13T17:43:38.321323+0000","last_became_active":"2024-09-13T17:43:36.975597+0000","last_became_peered":"2024-09-13T17:43:36.975597+0000","last_unstale":"2024-09-13T17:43:38.321323+0000","last_undegraded":"2024-09-13T17:43:38.321323+0000","last_fullsized":"2024-09-13T17:43:38.321323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:43.031842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321204+0000","last_change":"2024-09-13T17:43:36.981632+0000","last_active":"2024-09-13T17:43:38.321204+0000","last_peered":"2024-09-13T17:43:38.321204+0000","last_clean":"2024-09-13T17:43:38.321204+0000","last_became_active":"2024-09-13T17:43:36.977775+0000","last_became_peered":"2024-09-13T17:43:36.977775+0000","last_unstale":"2024-09-13T17:43:38.321204+0000","last_undegraded":"2024-09-13T17:43:38.321204+0000","last_fullsized":"2024-09-13T17:43:38.321204+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272348+0000","last_change":"2024-09-13T17:43:36.995318+0000","last_active":"2024-09-13T17:43:38.272348+0000","last_peered":"2024-09-13T17:43:38.272348+0000","last_clean":"2024-09-13T17:43:38.272348+0000","last_became_active":"2024-09-13T17:43:36.995041+0000","last_became_peered":"2024-09-13T17:43:36.995041+0000","last_unstale":"2024-09-13T17:43:38.272348+0000","last_undegraded":"2024-09-13T17:43:38.272348+0000","last_fullsized":"2024-09-13T17:43:38.272348+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:59.884186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271912+0000","last_change":"2024-09-13T17:43:36.972925+0000","last_active":"2024-09-13T17:43:38.271912+0000","last_peered":"2024-09-13T17:43:38.271912+0000","last_clean":"2024-09-13T17:43:38.271912+0000","last_became_active":"2024-09-13T17:43:36.972388+0000","last_became_peered":"2024-09-13T17:43:36.972388+0000","last_unstale":"2024-09-13T17:43:38.271912+0000","last_undegraded":"2024-09-13T17:43:38.271912+0000","last_fullsized":"2024-09-13T17:43:38.271912+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:30.822850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320627+0000","last_change":"2024-09-13T17:43:36.989729+0000","last_active":"2024-09-13T17:43:38.320627+0000","last_peered":"2024-09-13T17:43:38.320627+0000","last_clean":"2024-09-13T17:43:38.320627+0000","last_became_active":"2024-09-13T17:43:36.989287+0000","last_became_peered":"2024-09-13T17:43:36.989287+0000","last_unstale":"2024-09-13T17:43:38.320627+0000","last_undegraded":"2024-09-13T17:43:38.320627+0000","last_fullsized":"2024-09-13T17:43:38.320627+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274099+0000","last_change":"2024-09-13T17:43:36.972052+0000","last_active":"2024-09-13T17:43:38.274099+0000","last_peered":"2024-09-13T17:43:38.274099+0000","last_clean":"2024-09-13T17:43:38.274099+0000","last_became_active":"2024-09-13T17:43:36.971235+0000","last_became_peered":"2024-09-13T17:43:36.971235+0000","last_unstale":"2024-09-13T17:43:38.274099+0000","last_undegraded":"2024-09-13T17:43:38.274099+0000","last_fullsized":"2024-09-13T17:43:38.274099+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:28.557515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321828+0000","last_change":"2024-09-13T17:43:36.995992+0000","last_active":"2024-09-13T17:43:38.321828+0000","last_peered":"2024-09-13T17:43:38.321828+0000","last_clean":"2024-09-13T17:43:38.321828+0000","last_became_active":"2024-09-13T17:43:36.995839+0000","last_became_peered":"2024-09-13T17:43:36.995839+0000","last_unstale":"2024-09-13T17:43:38.321828+0000","last_undegraded":"2024-09-13T17:43:38.321828+0000","last_fullsized":"2024-09-13T17:43:38.321828+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:43.285004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273034+0000","last_change":"2024-09-13T17:43:36.999236+0000","last_active":"2024-09-13T17:43:38.273034+0000","last_peered":"2024-09-13T17:43:38.273034+0000","last_clean":"2024-09-13T17:43:38.273034+0000","last_became_active":"2024-09-13T17:43:36.998617+0000","last_became_peered":"2024-09-13T17:43:36.998617+0000","last_unstale":"2024-09-13T17:43:38.273034+0000","last_undegraded":"2024-09-13T17:43:38.273034+0000","last_fullsized":"2024-09-13T17:43:38.273034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:43.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321605+0000","last_change":"2024-09-13T17:43:36.994453+0000","last_active":"2024-09-13T17:43:38.321605+0000","last_peered":"2024-09-13T17:43:38.321605+0000","last_clean":"2024-09-13T17:43:38.321605+0000","last_became_active":"2024-09-13T17:43:36.993574+0000","last_became_peered":"2024-09-13T17:43:36.993574+0000","last_unstale":"2024-09-13T17:43:38.321605+0000","last_undegraded":"2024-09-13T17:43:38.321605+0000","last_fullsized":"2024-09-13T17:43:38.321605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272616+0000","last_change":"2024-09-13T17:43:36.998865+0000","last_active":"2024-09-13T17:43:38.272616+0000","last_peered":"2024-09-13T17:43:38.272616+0000","last_clean":"2024-09-13T17:43:38.272616+0000","last_became_active":"2024-09-13T17:43:36.998611+0000","last_became_peered":"2024-09-13T17:43:36.998611+0000","last_unstale":"2024-09-13T17:43:38.272616+0000","last_undegraded":"2024-09-13T17:43:38.272616+0000","last_fullsized":"2024-09-13T17:43:38.272616+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:31.977360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320600+0000","last_change":"2024-09-13T17:43:36.967204+0000","last_active":"2024-09-13T17:43:38.320600+0000","last_peered":"2024-09-13T17:43:38.320600+0000","last_clean":"2024-09-13T17:43:38.320600+0000","last_became_active":"2024-09-13T17:43:36.966928+0000","last_became_peered":"2024-09-13T17:43:36.966928+0000","last_unstale":"2024-09-13T17:43:38.320600+0000","last_undegraded":"2024-09-13T17:43:38.320600+0000","last_fullsized":"2024-09-13T17:43:38.320600+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:58.179836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950226+0000","last_change":"2024-09-13T17:43:36.989315+0000","last_active":"2024-09-13T17:43:37.950226+0000","last_peered":"2024-09-13T17:43:37.950226+0000","last_clean":"2024-09-13T17:43:37.950226+0000","last_became_active":"2024-09-13T17:43:36.989070+0000","last_became_peered":"2024-09-13T17:43:36.989070+0000","last_unstale":"2024-09-13T17:43:37.950226+0000","last_undegraded":"2024-09-13T17:43:37.950226+0000","last_fullsized":"2024-09-13T17:43:37.950226+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320598+0000","last_change":"2024-09-13T17:43:36.990968+0000","last_active":"2024-09-13T17:43:38.320598+0000","last_peered":"2024-09-13T17:43:38.320598+0000","last_clean":"2024-09-13T17:43:38.320598+0000","last_became_active":"2024-09-13T17:43:36.990688+0000","last_became_peered":"2024-09-13T17:43:36.990688+0000","last_unstale":"2024-09-13T17:43:38.320598+0000","last_undegraded":"2024-09-13T17:43:38.320598+0000","last_fullsized":"2024-09-13T17:43:38.320598+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951319+0000","last_change":"2024-09-13T17:43:36.997889+0000","last_active":"2024-09-13T17:43:37.951319+0000","last_peered":"2024-09-13T17:43:37.951319+0000","last_clean":"2024-09-13T17:43:37.951319+0000","last_became_active":"2024-09-13T17:43:36.997671+0000","last_became_peered":"2024-09-13T17:43:36.997671+0000","last_unstale":"2024-09-13T17:43:37.951319+0000","last_undegraded":"2024-09-13T17:43:37.951319+0000","last_fullsized":"2024-09-13T17:43:37.951319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"36'4","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322431+0000","last_change":"2024-09-13T17:43:36.597308+0000","last_active":"2024-09-13T17:43:38.322431+0000","last_peered":"2024-09-13T17:43:38.322431+0000","last_clean":"2024-09-13T17:43:38.322431+0000","last_became_active":"2024-09-13T17:43:36.596591+0000","last_became_peered":"2024-09-13T17:43:36.596591+0000","last_unstale":"2024-09-13T17:43:38.322431+0000","last_undegraded":"2024-09-13T17:43:38.322431+0000","last_fullsized":"2024-09-13T17:43:38.322431+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:03.103037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274269+0000","last_change":"2024-09-13T17:43:36.995868+0000","last_active":"2024-09-13T17:43:38.274269+0000","last_peered":"2024-09-13T17:43:38.274269+0000","last_clean":"2024-09-13T17:43:38.274269+0000","last_became_active":"2024-09-13T17:43:36.995304+0000","last_became_peered":"2024-09-13T17:43:36.995304+0000","last_unstale":"2024-09-13T17:43:38.274269+0000","last_undegraded":"2024-09-13T17:43:38.274269+0000","last_fullsized":"2024-09-13T17:43:38.274269+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:40.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951524+0000","last_change":"2024-09-13T17:43:36.593051+0000","last_active":"2024-09-13T17:43:37.951524+0000","last_peered":"2024-09-13T17:43:37.951524+0000","last_clean":"2024-09-13T17:43:37.951524+0000","last_became_active":"2024-09-13T17:43:36.592210+0000","last_became_peered":"2024-09-13T17:43:36.592210+0000","last_unstale":"2024-09-13T17:43:37.951524+0000","last_undegraded":"2024-09-13T17:43:37.951524+0000","last_fullsized":"2024-09-13T17:43:37.951524+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:07.519036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321670+0000","last_change":"2024-09-13T17:43:36.994621+0000","last_active":"2024-09-13T17:43:38.321670+0000","last_peered":"2024-09-13T17:43:38.321670+0000","last_clean":"2024-09-13T17:43:38.321670+0000","last_became_active":"2024-09-13T17:43:36.993891+0000","last_became_peered":"2024-09-13T17:43:36.993891+0000","last_unstale":"2024-09-13T17:43:38.321670+0000","last_undegraded":"2024-09-13T17:43:38.321670+0000","last_fullsized":"2024-09-13T17:43:38.321670+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951704+0000","last_change":"2024-09-13T17:43:36.607525+0000","last_active":"2024-09-13T17:43:37.951704+0000","last_peered":"2024-09-13T17:43:37.951704+0000","last_clean":"2024-09-13T17:43:37.951704+0000","last_became_active":"2024-09-13T17:43:36.606688+0000","last_became_peered":"2024-09-13T17:43:36.606688+0000","last_unstale":"2024-09-13T17:43:37.951704+0000","last_undegraded":"2024-09-13T17:43:37.951704+0000","last_fullsized":"2024-09-13T17:43:37.951704+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:34.433256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951210+0000","last_change":"2024-09-13T17:43:36.987727+0000","last_active":"2024-09-13T17:43:37.951210+0000","last_peered":"2024-09-13T17:43:37.951210+0000","last_clean":"2024-09-13T17:43:37.951210+0000","last_became_active":"2024-09-13T17:43:36.987360+0000","last_became_peered":"2024-09-13T17:43:36.987360+0000","last_unstale":"2024-09-13T17:43:37.951210+0000","last_undegraded":"2024-09-13T17:43:37.951210+0000","last_fullsized":"2024-09-13T17:43:37.951210+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951049+0000","last_change":"2024-09-13T17:43:36.604117+0000","last_active":"2024-09-13T17:43:37.951049+0000","last_peered":"2024-09-13T17:43:37.951049+0000","last_clean":"2024-09-13T17:43:37.951049+0000","last_became_active":"2024-09-13T17:43:36.603754+0000","last_became_peered":"2024-09-13T17:43:36.603754+0000","last_unstale":"2024-09-13T17:43:37.951049+0000","last_undegraded":"2024-09-13T17:43:37.951049+0000","last_fullsized":"2024-09-13T17:43:37.951049+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:48.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321117+0000","last_change":"2024-09-13T17:43:36.976896+0000","last_active":"2024-09-13T17:43:38.321117+0000","last_peered":"2024-09-13T17:43:38.321117+0000","last_clean":"2024-09-13T17:43:38.321117+0000","last_became_active":"2024-09-13T17:43:36.974032+0000","last_became_peered":"2024-09-13T17:43:36.974032+0000","last_unstale":"2024-09-13T17:43:38.321117+0000","last_undegraded":"2024-09-13T17:43:38.321117+0000","last_fullsized":"2024-09-13T17:43:38.321117+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:52.403736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1b","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273233+0000","last_change":"2024-09-13T17:43:36.601888+0000","last_active":"2024-09-13T17:43:38.273233+0000","last_peered":"2024-09-13T17:43:38.273233+0000","last_clean":"2024-09-13T17:43:38.273233+0000","last_became_active":"2024-09-13T17:43:36.601588+0000","last_became_peered":"2024-09-13T17:43:36.601588+0000","last_unstale":"2024-09-13T17:43:38.273233+0000","last_undegraded":"2024-09-13T17:43:38.273233+0000","last_fullsized":"2024-09-13T17:43:38.273233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:05.159220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272580+0000","last_change":"2024-09-13T17:43:36.976787+0000","last_active":"2024-09-13T17:43:38.272580+0000","last_peered":"2024-09-13T17:43:38.272580+0000","last_clean":"2024-09-13T17:43:38.272580+0000","last_became_active":"2024-09-13T17:43:36.976205+0000","last_became_peered":"2024-09-13T17:43:36.976205+0000","last_unstale":"2024-09-13T17:43:38.272580+0000","last_undegraded":"2024-09-13T17:43:38.272580+0000","last_fullsized":"2024-09-13T17:43:38.272580+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:09.035544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272638+0000","last_change":"2024-09-13T17:43:36.612531+0000","last_active":"2024-09-13T17:43:38.272638+0000","last_peered":"2024-09-13T17:43:38.272638+0000","last_clean":"2024-09-13T17:43:38.272638+0000","last_became_active":"2024-09-13T17:43:36.610289+0000","last_became_peered":"2024-09-13T17:43:36.610289+0000","last_unstale":"2024-09-13T17:43:38.272638+0000","last_undegraded":"2024-09-13T17:43:38.272638+0000","last_fullsized":"2024-09-13T17:43:38.272638+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:09.724844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271583+0000","last_change":"2024-09-13T17:43:36.991160+0000","last_active":"2024-09-13T17:43:38.271583+0000","last_peered":"2024-09-13T17:43:38.271583+0000","last_clean":"2024-09-13T17:43:38.271583+0000","last_became_active":"2024-09-13T17:43:36.990271+0000","last_became_peered":"2024-09-13T17:43:36.990271+0000","last_unstale":"2024-09-13T17:43:38.271583+0000","last_undegraded":"2024-09-13T17:43:38.271583+0000","last_fullsized":"2024-09-13T17:43:38.271583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:03.041558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272381+0000","last_change":"2024-09-13T17:43:36.612265+0000","last_active":"2024-09-13T17:43:38.272381+0000","last_peered":"2024-09-13T17:43:38.272381+0000","last_clean":"2024-09-13T17:43:38.272381+0000","last_became_active":"2024-09-13T17:43:36.609463+0000","last_became_peered":"2024-09-13T17:43:36.609463+0000","last_unstale":"2024-09-13T17:43:38.272381+0000","last_undegraded":"2024-09-13T17:43:38.272381+0000","last_fullsized":"2024-09-13T17:43:38.272381+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:51.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950545+0000","last_change":"2024-09-13T17:43:36.988557+0000","last_active":"2024-09-13T17:43:37.950545+0000","last_peered":"2024-09-13T17:43:37.950545+0000","last_clean":"2024-09-13T17:43:37.950545+0000","last_became_active":"2024-09-13T17:43:36.985705+0000","last_became_peered":"2024-09-13T17:43:36.985705+0000","last_unstale":"2024-09-13T17:43:37.950545+0000","last_undegraded":"2024-09-13T17:43:37.950545+0000","last_fullsized":"2024-09-13T17:43:37.950545+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271669+0000","last_change":"2024-09-13T17:43:36.614029+0000","last_active":"2024-09-13T17:43:38.271669+0000","last_peered":"2024-09-13T17:43:38.271669+0000","last_clean":"2024-09-13T17:43:38.271669+0000","last_became_active":"2024-09-13T17:43:36.612938+0000","last_became_peered":"2024-09-13T17:43:36.612938+0000","last_unstale":"2024-09-13T17:43:38.271669+0000","last_undegraded":"2024-09-13T17:43:38.271669+0000","last_fullsized":"2024-09-13T17:43:38.271669+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:57.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951416+0000","last_change":"2024-09-13T17:43:36.994355+0000","last_active":"2024-09-13T17:43:37.951416+0000","last_peered":"2024-09-13T17:43:37.951416+0000","last_clean":"2024-09-13T17:43:37.951416+0000","last_became_active":"2024-09-13T17:43:36.988868+0000","last_became_peered":"2024-09-13T17:43:36.988868+0000","last_unstale":"2024-09-13T17:43:37.951416+0000","last_undegraded":"2024-09-13T17:43:37.951416+0000","last_fullsized":"2024-09-13T17:43:37.951416+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951862+0000","last_change":"2024-09-13T17:43:36.606733+0000","last_active":"2024-09-13T17:43:37.951862+0000","last_peered":"2024-09-13T17:43:37.951862+0000","last_clean":"2024-09-13T17:43:37.951862+0000","last_became_active":"2024-09-13T17:43:36.604421+0000","last_became_peered":"2024-09-13T17:43:36.604421+0000","last_unstale":"2024-09-13T17:43:37.951862+0000","last_undegraded":"2024-09-13T17:43:37.951862+0000","last_fullsized":"2024-09-13T17:43:37.951862+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:21.923719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951610+0000","last_change":"2024-09-13T17:43:36.989522+0000","last_active":"2024-09-13T17:43:37.951610+0000","last_peered":"2024-09-13T17:43:37.951610+0000","last_clean":"2024-09-13T17:43:37.951610+0000","last_became_active":"2024-09-13T17:43:36.987659+0000","last_became_peered":"2024-09-13T17:43:36.987659+0000","last_unstale":"2024-09-13T17:43:37.951610+0000","last_undegraded":"2024-09-13T17:43:37.951610+0000","last_fullsized":"2024-09-13T17:43:37.951610+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274039+0000","last_change":"2024-09-13T17:43:36.610299+0000","last_active":"2024-09-13T17:43:38.274039+0000","last_peered":"2024-09-13T17:43:38.274039+0000","last_clean":"2024-09-13T17:43:38.274039+0000","last_became_active":"2024-09-13T17:43:36.609995+0000","last_became_peered":"2024-09-13T17:43:36.609995+0000","last_unstale":"2024-09-13T17:43:38.274039+0000","last_undegraded":"2024-09-13T17:43:38.274039+0000","last_fullsized":"2024-09-13T17:43:38.274039+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:06.509485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950972+0000","last_change":"2024-09-13T17:43:36.988822+0000","last_active":"2024-09-13T17:43:37.950972+0000","last_peered":"2024-09-13T17:43:37.950972+0000","last_clean":"2024-09-13T17:43:37.950972+0000","last_became_active":"2024-09-13T17:43:36.987189+0000","last_became_peered":"2024-09-13T17:43:36.987189+0000","last_unstale":"2024-09-13T17:43:37.950972+0000","last_undegraded":"2024-09-13T17:43:37.950972+0000","last_fullsized":"2024-09-13T17:43:37.950972+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"36'3","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321535+0000","last_change":"2024-09-13T17:43:36.604426+0000","last_active":"2024-09-13T17:43:38.321535+0000","last_peered":"2024-09-13T17:43:38.321535+0000","last_clean":"2024-09-13T17:43:38.321535+0000","last_became_active":"2024-09-13T17:43:36.603659+0000","last_became_peered":"2024-09-13T17:43:36.603659+0000","last_unstale":"2024-09-13T17:43:38.321535+0000","last_undegraded":"2024-09-13T17:43:38.321535+0000","last_fullsized":"2024-09-13T17:43:38.321535+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:27.409650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321377+0000","last_change":"2024-09-13T17:43:36.991166+0000","last_active":"2024-09-13T17:43:38.321377+0000","last_peered":"2024-09-13T17:43:38.321377+0000","last_clean":"2024-09-13T17:43:38.321377+0000","last_became_active":"2024-09-13T17:43:36.990864+0000","last_became_peered":"2024-09-13T17:43:36.990864+0000","last_unstale":"2024-09-13T17:43:38.321377+0000","last_undegraded":"2024-09-13T17:43:38.321377+0000","last_fullsized":"2024-09-13T17:43:38.321377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"36'6","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322085+0000","last_change":"2024-09-13T17:43:36.603712+0000","last_active":"2024-09-13T17:43:38.322085+0000","last_peered":"2024-09-13T17:43:38.322085+0000","last_clean":"2024-09-13T17:43:38.322085+0000","last_became_active":"2024-09-13T17:43:36.603454+0000","last_became_peered":"2024-09-13T17:43:36.603454+0000","last_unstale":"2024-09-13T17:43:38.322085+0000","last_undegraded":"2024-09-13T17:43:38.322085+0000","last_fullsized":"2024-09-13T17:43:38.322085+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:01.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950478+0000","last_change":"2024-09-13T17:43:36.988758+0000","last_active":"2024-09-13T17:43:37.950478+0000","last_peered":"2024-09-13T17:43:37.950478+0000","last_clean":"2024-09-13T17:43:37.950478+0000","last_became_active":"2024-09-13T17:43:36.987064+0000","last_became_peered":"2024-09-13T17:43:36.987064+0000","last_unstale":"2024-09-13T17:43:37.950478+0000","last_undegraded":"2024-09-13T17:43:37.950478+0000","last_fullsized":"2024-09-13T17:43:37.950478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321328+0000","last_change":"2024-09-13T17:43:36.590347+0000","last_active":"2024-09-13T17:43:38.321328+0000","last_peered":"2024-09-13T17:43:38.321328+0000","last_clean":"2024-09-13T17:43:38.321328+0000","last_became_active":"2024-09-13T17:43:36.589982+0000","last_became_peered":"2024-09-13T17:43:36.589982+0000","last_unstale":"2024-09-13T17:43:38.321328+0000","last_undegraded":"2024-09-13T17:43:38.321328+0000","last_fullsized":"2024-09-13T17:43:38.321328+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:03.593342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273401+0000","last_change":"2024-09-13T17:43:36.976611+0000","last_active":"2024-09-13T17:43:38.273401+0000","last_peered":"2024-09-13T17:43:38.273401+0000","last_clean":"2024-09-13T17:43:38.273401+0000","last_became_active":"2024-09-13T17:43:36.972871+0000","last_became_peered":"2024-09-13T17:43:36.972871+0000","last_unstale":"2024-09-13T17:43:38.273401+0000","last_undegraded":"2024-09-13T17:43:38.273401+0000","last_fullsized":"2024-09-13T17:43:38.273401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:02.818469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272719+0000","last_change":"2024-09-13T17:43:36.602194+0000","last_active":"2024-09-13T17:43:38.272719+0000","last_peered":"2024-09-13T17:43:38.272719+0000","last_clean":"2024-09-13T17:43:38.272719+0000","last_became_active":"2024-09-13T17:43:36.601939+0000","last_became_peered":"2024-09-13T17:43:36.601939+0000","last_unstale":"2024-09-13T17:43:38.272719+0000","last_undegraded":"2024-09-13T17:43:38.272719+0000","last_fullsized":"2024-09-13T17:43:38.272719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:32.706151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950799+0000","last_change":"2024-09-13T17:43:36.980350+0000","last_active":"2024-09-13T17:43:37.950799+0000","last_peered":"2024-09-13T17:43:37.950799+0000","last_clean":"2024-09-13T17:43:37.950799+0000","last_became_active":"2024-09-13T17:43:36.977778+0000","last_became_peered":"2024-09-13T17:43:36.977778+0000","last_unstale":"2024-09-13T17:43:37.950799+0000","last_undegraded":"2024-09-13T17:43:37.950799+0000","last_fullsized":"2024-09-13T17:43:37.950799+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320658+0000","last_change":"2024-09-13T17:43:36.597400+0000","last_active":"2024-09-13T17:43:38.320658+0000","last_peered":"2024-09-13T17:43:38.320658+0000","last_clean":"2024-09-13T17:43:38.320658+0000","last_became_active":"2024-09-13T17:43:36.596736+0000","last_became_peered":"2024-09-13T17:43:36.596736+0000","last_unstale":"2024-09-13T17:43:38.320658+0000","last_undegraded":"2024-09-13T17:43:38.320658+0000","last_fullsized":"2024-09-13T17:43:38.320658+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:21.792293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320430+0000","last_change":"2024-09-13T17:43:36.967452+0000","last_active":"2024-09-13T17:43:38.320430+0000","last_peered":"2024-09-13T17:43:38.320430+0000","last_clean":"2024-09-13T17:43:38.320430+0000","last_became_active":"2024-09-13T17:43:36.966901+0000","last_became_peered":"2024-09-13T17:43:36.966901+0000","last_unstale":"2024-09-13T17:43:38.320430+0000","last_undegraded":"2024-09-13T17:43:38.320430+0000","last_fullsized":"2024-09-13T17:43:38.320430+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"36'8","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271906+0000","last_change":"2024-09-13T17:43:36.607932+0000","last_active":"2024-09-13T17:43:38.271906+0000","last_peered":"2024-09-13T17:43:38.271906+0000","last_clean":"2024-09-13T17:43:38.271906+0000","last_became_active":"2024-09-13T17:43:36.607651+0000","last_became_peered":"2024-09-13T17:43:36.607651+0000","last_unstale":"2024-09-13T17:43:38.271906+0000","last_undegraded":"2024-09-13T17:43:38.271906+0000","last_fullsized":"2024-09-13T17:43:38.271906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:09.002010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321981+0000","last_change":"2024-09-13T17:43:36.996236+0000","last_active":"2024-09-13T17:43:38.321981+0000","last_peered":"2024-09-13T17:43:38.321981+0000","last_clean":"2024-09-13T17:43:38.321981+0000","last_became_active":"2024-09-13T17:43:36.995955+0000","last_became_peered":"2024-09-13T17:43:36.995955+0000","last_unstale":"2024-09-13T17:43:38.321981+0000","last_undegraded":"2024-09-13T17:43:38.321981+0000","last_fullsized":"2024-09-13T17:43:38.321981+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:23.137553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.322377+0000","last_change":"2024-09-13T17:43:36.602229+0000","last_active":"2024-09-13T17:43:38.322377+0000","last_peered":"2024-09-13T17:43:38.322377+0000","last_clean":"2024-09-13T17:43:38.322377+0000","last_became_active":"2024-09-13T17:43:36.601122+0000","last_became_peered":"2024-09-13T17:43:36.601122+0000","last_unstale":"2024-09-13T17:43:38.322377+0000","last_undegraded":"2024-09-13T17:43:38.322377+0000","last_fullsized":"2024-09-13T17:43:38.322377+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:25.802228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274309+0000","last_change":"2024-09-13T17:43:36.996063+0000","last_active":"2024-09-13T17:43:38.274309+0000","last_peered":"2024-09-13T17:43:38.274309+0000","last_clean":"2024-09-13T17:43:38.274309+0000","last_became_active":"2024-09-13T17:43:36.995621+0000","last_became_peered":"2024-09-13T17:43:36.995621+0000","last_unstale":"2024-09-13T17:43:38.274309+0000","last_undegraded":"2024-09-13T17:43:38.274309+0000","last_fullsized":"2024-09-13T17:43:38.274309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:12.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951614+0000","last_change":"2024-09-13T17:43:36.594362+0000","last_active":"2024-09-13T17:43:37.951614+0000","last_peered":"2024-09-13T17:43:37.951614+0000","last_clean":"2024-09-13T17:43:37.951614+0000","last_became_active":"2024-09-13T17:43:36.593113+0000","last_became_peered":"2024-09-13T17:43:36.593113+0000","last_unstale":"2024-09-13T17:43:37.951614+0000","last_undegraded":"2024-09-13T17:43:37.951614+0000","last_fullsized":"2024-09-13T17:43:37.951614+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:01.736566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321882+0000","last_change":"2024-09-13T17:43:36.994777+0000","last_active":"2024-09-13T17:43:38.321882+0000","last_peered":"2024-09-13T17:43:38.321882+0000","last_clean":"2024-09-13T17:43:38.321882+0000","last_became_active":"2024-09-13T17:43:36.994213+0000","last_became_peered":"2024-09-13T17:43:36.994213+0000","last_unstale":"2024-09-13T17:43:38.321882+0000","last_undegraded":"2024-09-13T17:43:38.321882+0000","last_fullsized":"2024-09-13T17:43:38.321882+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"36'1","reported_seq":12,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951755+0000","last_change":"2024-09-13T17:43:36.607688+0000","last_active":"2024-09-13T17:43:37.951755+0000","last_peered":"2024-09-13T17:43:37.951755+0000","last_clean":"2024-09-13T17:43:37.951755+0000","last_became_active":"2024-09-13T17:43:36.607290+0000","last_became_peered":"2024-09-13T17:43:36.607290+0000","last_unstale":"2024-09-13T17:43:37.951755+0000","last_undegraded":"2024-09-13T17:43:37.951755+0000","last_fullsized":"2024-09-13T17:43:37.951755+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:24.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951500+0000","last_change":"2024-09-13T17:43:36.987379+0000","last_active":"2024-09-13T17:43:37.951500+0000","last_peered":"2024-09-13T17:43:37.951500+0000","last_clean":"2024-09-13T17:43:37.951500+0000","last_became_active":"2024-09-13T17:43:36.986990+0000","last_became_peered":"2024-09-13T17:43:36.986990+0000","last_unstale":"2024-09-13T17:43:37.951500+0000","last_undegraded":"2024-09-13T17:43:37.951500+0000","last_fullsized":"2024-09-13T17:43:37.951500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951146+0000","last_change":"2024-09-13T17:43:36.596481+0000","last_active":"2024-09-13T17:43:37.951146+0000","last_peered":"2024-09-13T17:43:37.951146+0000","last_clean":"2024-09-13T17:43:37.951146+0000","last_became_active":"2024-09-13T17:43:36.594330+0000","last_became_peered":"2024-09-13T17:43:36.594330+0000","last_unstale":"2024-09-13T17:43:37.951146+0000","last_undegraded":"2024-09-13T17:43:37.951146+0000","last_fullsized":"2024-09-13T17:43:37.951146+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:52.485090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321058+0000","last_change":"2024-09-13T17:43:36.977463+0000","last_active":"2024-09-13T17:43:38.321058+0000","last_peered":"2024-09-13T17:43:38.321058+0000","last_clean":"2024-09-13T17:43:38.321058+0000","last_became_active":"2024-09-13T17:43:36.977207+0000","last_became_peered":"2024-09-13T17:43:36.977207+0000","last_unstale":"2024-09-13T17:43:38.321058+0000","last_undegraded":"2024-09-13T17:43:38.321058+0000","last_fullsized":"2024-09-13T17:43:38.321058+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:02:51.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951408+0000","last_change":"2024-09-13T17:43:36.607315+0000","last_active":"2024-09-13T17:43:37.951408+0000","last_peered":"2024-09-13T17:43:37.951408+0000","last_clean":"2024-09-13T17:43:37.951408+0000","last_became_active":"2024-09-13T17:43:36.606824+0000","last_became_peered":"2024-09-13T17:43:36.606824+0000","last_unstale":"2024-09-13T17:43:37.951408+0000","last_undegraded":"2024-09-13T17:43:37.951408+0000","last_fullsized":"2024-09-13T17:43:37.951408+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.362407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950605+0000","last_change":"2024-09-13T17:43:36.991670+0000","last_active":"2024-09-13T17:43:37.950605+0000","last_peered":"2024-09-13T17:43:37.950605+0000","last_clean":"2024-09-13T17:43:37.950605+0000","last_became_active":"2024-09-13T17:43:36.987763+0000","last_became_peered":"2024-09-13T17:43:36.987763+0000","last_unstale":"2024-09-13T17:43:37.950605+0000","last_undegraded":"2024-09-13T17:43:37.950605+0000","last_fullsized":"2024-09-13T17:43:37.950605+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272743+0000","last_change":"2024-09-13T17:43:36.612138+0000","last_active":"2024-09-13T17:43:38.272743+0000","last_peered":"2024-09-13T17:43:38.272743+0000","last_clean":"2024-09-13T17:43:38.272743+0000","last_became_active":"2024-09-13T17:43:36.609873+0000","last_became_peered":"2024-09-13T17:43:36.609873+0000","last_unstale":"2024-09-13T17:43:38.272743+0000","last_undegraded":"2024-09-13T17:43:38.272743+0000","last_fullsized":"2024-09-13T17:43:38.272743+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:49.578767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320804+0000","last_change":"2024-09-13T17:43:36.990056+0000","last_active":"2024-09-13T17:43:38.320804+0000","last_peered":"2024-09-13T17:43:38.320804+0000","last_clean":"2024-09-13T17:43:38.320804+0000","last_became_active":"2024-09-13T17:43:36.989640+0000","last_became_peered":"2024-09-13T17:43:36.989640+0000","last_unstale":"2024-09-13T17:43:38.320804+0000","last_undegraded":"2024-09-13T17:43:38.320804+0000","last_fullsized":"2024-09-13T17:43:38.320804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272372+0000","last_change":"2024-09-13T17:43:36.612085+0000","last_active":"2024-09-13T17:43:38.272372+0000","last_peered":"2024-09-13T17:43:38.272372+0000","last_clean":"2024-09-13T17:43:38.272372+0000","last_became_active":"2024-09-13T17:43:36.609135+0000","last_became_peered":"2024-09-13T17:43:36.609135+0000","last_unstale":"2024-09-13T17:43:38.272372+0000","last_undegraded":"2024-09-13T17:43:38.272372+0000","last_fullsized":"2024-09-13T17:43:38.272372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:22.346224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271989+0000","last_change":"2024-09-13T17:43:36.999734+0000","last_active":"2024-09-13T17:43:38.271989+0000","last_peered":"2024-09-13T17:43:38.271989+0000","last_clean":"2024-09-13T17:43:38.271989+0000","last_became_active":"2024-09-13T17:43:36.999416+0000","last_became_peered":"2024-09-13T17:43:36.999416+0000","last_unstale":"2024-09-13T17:43:38.271989+0000","last_undegraded":"2024-09-13T17:43:38.271989+0000","last_fullsized":"2024-09-13T17:43:38.271989+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:11.465274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320850+0000","last_change":"2024-09-13T17:43:36.598625+0000","last_active":"2024-09-13T17:43:38.320850+0000","last_peered":"2024-09-13T17:43:38.320850+0000","last_clean":"2024-09-13T17:43:38.320850+0000","last_became_active":"2024-09-13T17:43:36.598229+0000","last_became_peered":"2024-09-13T17:43:36.598229+0000","last_unstale":"2024-09-13T17:43:38.320850+0000","last_undegraded":"2024-09-13T17:43:38.320850+0000","last_fullsized":"2024-09-13T17:43:38.320850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:54.846315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951787+0000","last_change":"2024-09-13T17:43:36.988413+0000","last_active":"2024-09-13T17:43:37.951787+0000","last_peered":"2024-09-13T17:43:37.951787+0000","last_clean":"2024-09-13T17:43:37.951787+0000","last_became_active":"2024-09-13T17:43:36.987084+0000","last_became_peered":"2024-09-13T17:43:36.987084+0000","last_unstale":"2024-09-13T17:43:37.951787+0000","last_undegraded":"2024-09-13T17:43:37.951787+0000","last_fullsized":"2024-09-13T17:43:37.951787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951602+0000","last_change":"2024-09-13T17:43:36.596762+0000","last_active":"2024-09-13T17:43:37.951602+0000","last_peered":"2024-09-13T17:43:37.951602+0000","last_clean":"2024-09-13T17:43:37.951602+0000","last_became_active":"2024-09-13T17:43:36.594514+0000","last_became_peered":"2024-09-13T17:43:36.594514+0000","last_unstale":"2024-09-13T17:43:37.951602+0000","last_undegraded":"2024-09-13T17:43:37.951602+0000","last_fullsized":"2024-09-13T17:43:37.951602+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:30:50.353395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273094+0000","last_change":"2024-09-13T17:43:36.999031+0000","last_active":"2024-09-13T17:43:38.273094+0000","last_peered":"2024-09-13T17:43:38.273094+0000","last_clean":"2024-09-13T17:43:38.273094+0000","last_became_active":"2024-09-13T17:43:36.998251+0000","last_became_peered":"2024-09-13T17:43:36.998251+0000","last_unstale":"2024-09-13T17:43:38.273094+0000","last_undegraded":"2024-09-13T17:43:38.273094+0000","last_fullsized":"2024-09-13T17:43:38.273094+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:32.921571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273585+0000","last_change":"2024-09-13T17:43:36.612328+0000","last_active":"2024-09-13T17:43:38.273585+0000","last_peered":"2024-09-13T17:43:38.273585+0000","last_clean":"2024-09-13T17:43:38.273585+0000","last_became_active":"2024-09-13T17:43:36.610325+0000","last_became_peered":"2024-09-13T17:43:36.610325+0000","last_unstale":"2024-09-13T17:43:38.273585+0000","last_undegraded":"2024-09-13T17:43:38.273585+0000","last_fullsized":"2024-09-13T17:43:38.273585+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:31.356170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321382+0000","last_change":"2024-09-13T17:43:36.973857+0000","last_active":"2024-09-13T17:43:38.321382+0000","last_peered":"2024-09-13T17:43:38.321382+0000","last_clean":"2024-09-13T17:43:38.321382+0000","last_became_active":"2024-09-13T17:43:36.973595+0000","last_became_peered":"2024-09-13T17:43:36.973595+0000","last_unstale":"2024-09-13T17:43:38.321382+0000","last_undegraded":"2024-09-13T17:43:38.321382+0000","last_fullsized":"2024-09-13T17:43:38.321382+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:06.165795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.951161+0000","last_change":"2024-09-13T17:43:36.592618+0000","last_active":"2024-09-13T17:43:37.951161+0000","last_peered":"2024-09-13T17:43:37.951161+0000","last_clean":"2024-09-13T17:43:37.951161+0000","last_became_active":"2024-09-13T17:43:36.592303+0000","last_became_peered":"2024-09-13T17:43:36.592303+0000","last_unstale":"2024-09-13T17:43:37.951161+0000","last_undegraded":"2024-09-13T17:43:37.951161+0000","last_fullsized":"2024-09-13T17:43:37.951161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:16.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.321211+0000","last_change":"2024-09-13T17:43:36.994616+0000","last_active":"2024-09-13T17:43:38.321211+0000","last_peered":"2024-09-13T17:43:38.321211+0000","last_clean":"2024-09-13T17:43:38.321211+0000","last_became_active":"2024-09-13T17:43:36.994356+0000","last_became_peered":"2024-09-13T17:43:36.994356+0000","last_unstale":"2024-09-13T17:43:38.321211+0000","last_undegraded":"2024-09-13T17:43:38.321211+0000","last_fullsized":"2024-09-13T17:43:38.321211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273571+0000","last_change":"2024-09-13T17:43:36.601819+0000","last_active":"2024-09-13T17:43:38.273571+0000","last_peered":"2024-09-13T17:43:38.273571+0000","last_clean":"2024-09-13T17:43:38.273571+0000","last_became_active":"2024-09-13T17:43:36.601091+0000","last_became_peered":"2024-09-13T17:43:36.601091+0000","last_unstale":"2024-09-13T17:43:38.273571+0000","last_undegraded":"2024-09-13T17:43:38.273571+0000","last_fullsized":"2024-09-13T17:43:38.273571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:22.130979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.272856+0000","last_change":"2024-09-13T17:43:36.976449+0000","last_active":"2024-09-13T17:43:38.272856+0000","last_peered":"2024-09-13T17:43:38.272856+0000","last_clean":"2024-09-13T17:43:38.272856+0000","last_became_active":"2024-09-13T17:43:36.972460+0000","last_became_peered":"2024-09-13T17:43:36.972460+0000","last_unstale":"2024-09-13T17:43:38.272856+0000","last_undegraded":"2024-09-13T17:43:38.272856+0000","last_fullsized":"2024-09-13T17:43:38.272856+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:12.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950576+0000","last_change":"2024-09-13T17:43:36.608221+0000","last_active":"2024-09-13T17:43:37.950576+0000","last_peered":"2024-09-13T17:43:37.950576+0000","last_clean":"2024-09-13T17:43:37.950576+0000","last_became_active":"2024-09-13T17:43:36.607779+0000","last_became_peered":"2024-09-13T17:43:36.607779+0000","last_unstale":"2024-09-13T17:43:37.950576+0000","last_undegraded":"2024-09-13T17:43:37.950576+0000","last_fullsized":"2024-09-13T17:43:37.950576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:00.401036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950260+0000","last_change":"2024-09-13T17:43:36.977858+0000","last_active":"2024-09-13T17:43:37.950260+0000","last_peered":"2024-09-13T17:43:37.950260+0000","last_clean":"2024-09-13T17:43:37.950260+0000","last_became_active":"2024-09-13T17:43:36.977282+0000","last_became_peered":"2024-09-13T17:43:36.977282+0000","last_unstale":"2024-09-13T17:43:37.950260+0000","last_undegraded":"2024-09-13T17:43:37.950260+0000","last_fullsized":"2024-09-13T17:43:37.950260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:37.950612+0000","last_change":"2024-09-13T17:43:36.601601+0000","last_active":"2024-09-13T17:43:37.950612+0000","last_peered":"2024-09-13T17:43:37.950612+0000","last_clean":"2024-09-13T17:43:37.950612+0000","last_became_active":"2024-09-13T17:43:36.594738+0000","last_became_peered":"2024-09-13T17:43:36.594738+0000","last_unstale":"2024-09-13T17:43:37.950612+0000","last_undegraded":"2024-09-13T17:43:37.950612+0000","last_fullsized":"2024-09-13T17:43:37.950612+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:26.235235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.320493+0000","last_change":"2024-09-13T17:43:36.990024+0000","last_active":"2024-09-13T17:43:38.320493+0000","last_peered":"2024-09-13T17:43:38.320493+0000","last_clean":"2024-09-13T17:43:38.320493+0000","last_became_active":"2024-09-13T17:43:36.989774+0000","last_became_peered":"2024-09-13T17:43:36.989774+0000","last_unstale":"2024-09-13T17:43:38.320493+0000","last_undegraded":"2024-09-13T17:43:38.320493+0000","last_fullsized":"2024-09-13T17:43:38.320493+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.271761+0000","last_change":"2024-09-13T17:43:36.608307+0000","last_active":"2024-09-13T17:43:38.271761+0000","last_peered":"2024-09-13T17:43:38.271761+0000","last_clean":"2024-09-13T17:43:38.271761+0000","last_became_active":"2024-09-13T17:43:36.607846+0000","last_became_peered":"2024-09-13T17:43:36.607846+0000","last_unstale":"2024-09-13T17:43:38.271761+0000","last_undegraded":"2024-09-13T17:43:38.271761+0000","last_fullsized":"2024-09-13T17:43:38.271761+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:34.779723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.273934+0000","last_change":"2024-09-13T17:43:36.972148+0000","last_active":"2024-09-13T17:43:38.273934+0000","last_peered":"2024-09-13T17:43:38.273934+0000","last_clean":"2024-09-13T17:43:38.273934+0000","last_became_active":"2024-09-13T17:43:36.971337+0000","last_became_peered":"2024-09-13T17:43:36.971337+0000","last_unstale":"2024-09-13T17:43:38.273934+0000","last_undegraded":"2024-09-13T17:43:38.273934+0000","last_fullsized":"2024-09-13T17:43:38.273934+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:15.285058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"36'2","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-13T17:43:38.274593+0000","last_change":"2024-09-13T17:43:36.601639+0000","last_active":"2024-09-13T17:43:38.274593+0000","last_peered":"2024-09-13T17:43:38.274593+0000","last_clean":"2024-09-13T17:43:38.274593+0000","last_became_active":"2024-09-13T17:43:36.601329+0000","last_became_peered":"2024-09-13T17:43:36.601329+0000","last_unstale":"2024-09-13T17:43:38.274593+0000","last_undegraded":"2024-09-13T17:43:38.274593+0000","last_fullsized":"2024-09-13T17:43:38.274593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:43.542629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":239,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-13T17:43:36.953347+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:43:36.953347+0000","last_peered":"2024-09-13T17:43:36.953347+0000","last_clean":"2024-09-13T17:43:36.953347+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:43:36.953347+0000","last_undegraded":"2024-09-13T17:43:36.953347+0000","last_fullsized":"2024-09-13T17:43:36.953347+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":177,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":298,"num_read_kb":503,"num_write":177,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738392,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":125763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68100000000000005}]}]},{"osd":1,"up_from":8,"seq":34359738393,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":933888,"data_stored":713965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.504},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.628}]},{"osd":2,"last update":"Fri Sep 13 17:42:53 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.72399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":937984,"data_stored":712115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.59699999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:42:51 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:43:40.244 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:43:40.244 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:43:40.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f596cf640 1 Processor -- start 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f596cf640 1 -- start start 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f596cf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2f54138080 0x7f2f54138460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f596cf640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f540a2290 con 0x7f2f541389a0 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f596cf640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5406a8d0 con 0x7f2f54138080 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f52ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2f54138080 0x7f2f54138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f52ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2f54138080 0x7f2f54138460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50408/0 (socket says 172.21.15.96:50408) 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f52ffd640 1 -- 172.21.15.96:0/300883641 learned_addr learned my addr 172.21.15.96:0/300883641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f52ffd640 1 -- 172.21.15.96:0/300883641 >> v1:172.21.15.96:6789/0 conn(0x7f2f541389a0 legacy=0x7f2f5409ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.401+0000 7f2f52ffd640 1 -- 172.21.15.96:0/300883641 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f540a2e00 con 0x7f2f54138080 2024-09-13T17:43:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f52ffd640 1 --2- 172.21.15.96:0/300883641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f54138460 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2f440097f0 tx=0x7f2f4401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ad343f13d00937 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f51ffb640 1 -- 172.21.15.96:0/300883641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f440ab530 con 0x7f2f54138080 2024-09-13T17:43:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f51ffb640 1 -- 172.21.15.96:0/300883641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440a7050 con 0x7f2f54138080 2024-09-13T17:43:40.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f51ffb640 1 -- 172.21.15.96:0/300883641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f440ab7e0 con 0x7f2f54138080 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- 172.21.15.96:0/300883641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 msgr2=0x7f2f54138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 --2- 172.21.15.96:0/300883641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f54138460 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2f440097f0 tx=0x7f2f4401c450 comp rx=0 tx=0).stop 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- 172.21.15.96:0/300883641 shutdown_connections 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 --2- 172.21.15.96:0/300883641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f54138460 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- 172.21.15.96:0/300883641 >> 172.21.15.96:0/300883641 conn(0x7f2f540a7390 msgr2=0x7f2f540a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- 172.21.15.96:0/300883641 shutdown_connections 2024-09-13T17:43:40.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- 172.21.15.96:0/300883641 wait complete. 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 Processor -- start 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.405+0000 7f2f596cf640 1 -- start start 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f596cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f596cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5406e6e0 con 0x7f2f54138080 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50412/0 (socket says 172.21.15.96:50412) 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f52ffd640 1 -- 172.21.15.96:0/4176783733 learned_addr learned my addr 172.21.15.96:0/4176783733 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f52ffd640 1 -- 172.21.15.96:0/4176783733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f440094d0 con 0x7f2f54138080 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f52ffd640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2f4401a080 tx=0x7f2f4401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f440b6630 con 0x7f2f54138080 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f440b6790 con 0x7f2f54138080 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f54070eb0 con 0x7f2f54138080 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2f440b6a50 con 0x7f2f54138080 2024-09-13T17:43:40.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f54071cd0 con 0x7f2f54138080 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f2f440b2040 con 0x7f2f54138080 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5413d1c0 con 0x7f2f54138080 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 0x7f2f34037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f2f440e20c0 con 0x7f2f54138080 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f527fc640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 0x7f2f34037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.409+0000 7f2f527fc640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 0x7f2f34037a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f540a2d30 tx=0x7f2f3c0079e0 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.413+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2f440c7940 con 0x7f2f54138080 2024-09-13T17:43:40.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.613+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2f54138460 con 0x7f2f54138080 2024-09-13T17:43:40.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.617+0000 7f2f4b7fe640 1 -- 172.21.15.96:0/4176783733 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2f440b0020 con 0x7f2f54138080 2024-09-13T17:43:40.622 INFO:teuthology.orchestra.run.smithi096.stderr:updated caps for client.0 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 msgr2=0x7f2f34037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 0x7f2f34037a30 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2f540a2d30 tx=0x7f2f3c0079e0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 msgr2=0x7f2f5406ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2f4401a080 tx=0x7f2f4401ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 shutdown_connections 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f2f34035570 0x7f2f34037a30 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 --2- 172.21.15.96:0/4176783733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f54138080 0x7f2f5406ebd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 >> 172.21.15.96:0/4176783733 conn(0x7f2f540a7390 msgr2=0x7f2f54137260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 shutdown_connections 2024-09-13T17:43:40.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.621+0000 7f2f596cf640 1 -- 172.21.15.96:0/4176783733 wait complete. 2024-09-13T17:43:40.639 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:40.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7ddf4d640 1 Processor -- start 2024-09-13T17:43:40.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7ddf4d640 1 -- start start 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7ddf4d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa7d809d690 0x7fa7d8138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7ddf4d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d8138050 con 0x7fa7d81387d0 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7ddf4d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d809c410 con 0x7fa7d809d690 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa7d809d690 0x7fa7d8138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa7d81387d0 0x7fa7d8134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33894/0 (socket says 172.21.15.96:33894) 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d6ffd640 1 -- 172.21.15.96:0/747342808 learned_addr learned my addr 172.21.15.96:0/747342808 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d77fe640 1 -- 172.21.15.96:0/747342808 >> v1:172.21.15.96:6789/0 conn(0x7fa7d81387d0 legacy=0x7fa7d8134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d77fe640 1 -- 172.21.15.96:0/747342808 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d809cba0 con 0x7fa7d809d690 2024-09-13T17:43:40.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.749+0000 7fa7d67fc640 1 -- 172.21.15.96:0/747342808 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471213224 0 0) 0x7fa7c0001120 con 0x7fa7d81387d0 2024-09-13T17:43:40.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7d77fe640 1 --2- 172.21.15.96:0/747342808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d8138290 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009740 tx=0x7fa7c801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3374b0a3961f71d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7d67fc640 1 -- 172.21.15.96:0/747342808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7c80a8840 con 0x7fa7d809d690 2024-09-13T17:43:40.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7d67fc640 1 -- 172.21.15.96:0/747342808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c80a89a0 con 0x7fa7d809d690 2024-09-13T17:43:40.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7d67fc640 1 -- 172.21.15.96:0/747342808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7c80a8c60 con 0x7fa7d809d690 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/747342808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 msgr2=0x7fa7d8138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/747342808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d8138290 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009740 tx=0x7fa7c801ec10 comp rx=0 tx=0).stop 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/747342808 shutdown_connections 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/747342808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d8138290 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/747342808 >> 172.21.15.96:0/747342808 conn(0x7fa7d806bc60 msgr2=0x7fa7d806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/747342808 shutdown_connections 2024-09-13T17:43:40.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/747342808 wait complete. 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 Processor -- start 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.753+0000 7fa7ddf4d640 1 -- start start 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7ddf4d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7ddf4d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d80a5dd0 con 0x7fa7d809d690 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50438/0 (socket says 172.21.15.96:50438) 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d77fe640 1 -- 172.21.15.96:0/3592584747 learned_addr learned my addr 172.21.15.96:0/3592584747 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:40.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d77fe640 1 -- 172.21.15.96:0/3592584747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c8009420 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d77fe640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7c801c840 tx=0x7fa7c801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7c80b0840 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c80b09a0 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d80a60e0 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7c80b0c60 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d80a42a0 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fa7c80b7040 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 0x7fa7b8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d809bc00 con 0x7fa7d809d690 2024-09-13T17:43:40.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fa7c80e54d0 con 0x7fa7d809d690 2024-09-13T17:43:40.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d6ffd640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 0x7fa7b8037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:40.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.757+0000 7fa7d6ffd640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 0x7fa7b8037a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005890 tx=0x7fa7c0001280 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:40.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.761+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa7d809bc00 con 0x7fa7d809d690 2024-09-13T17:43:40.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7d809bc00 con 0x7fa7d809d690 2024-09-13T17:43:40.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7cffff640 1 -- 172.21.15.96:0/3592584747 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3217 (secure 0 0 0) 0x7fa7d809bc00 con 0x7fa7d809d690 2024-09-13T17:43:40.952 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:40.952 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4740,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-09-13T17:43:40.952 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 20 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 msgr2=0x7fa7b8037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 0x7fa7b8037a30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0005890 tx=0x7fa7c0001280 comp rx=0 tx=0).stop 2024-09-13T17:43:40.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 msgr2=0x7fa7d80a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa7c801c840 tx=0x7fa7c801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 shutdown_connections 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fa7b8035570 0x7fa7b8037a30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 --2- 172.21.15.96:0/3592584747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7d809d690 0x7fa7d80a6b20 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 >> 172.21.15.96:0/3592584747 conn(0x7fa7d806bc60 msgr2=0x7fa7d81bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 shutdown_connections 2024-09-13T17:43:40.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:40.949+0000 7fa7ddf4d640 1 -- 172.21.15.96:0/3592584747 wait complete. 2024-09-13T17:43:41.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:43:36.940692+0000', 'modified': '2024-09-13T17:43:38.946029+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4743}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4743': {'gid': 4743, 'name': 'b', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6829/2227270685', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 2227270685}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 2227270685}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:43:41.169 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:43:41.170 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:43:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518b582640 1 Processor -- start 2024-09-13T17:43:41.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518b582640 1 -- start start 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518b582640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f518409e820 0x7f518413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518b582640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51840a68b0 con 0x7f5184134e70 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518b582640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518409ddc0 con 0x7f518409e820 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f5188b1d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f518409e820 0x7f518413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518931e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5184134e70 0x7f5184135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33902/0 (socket says 172.21.15.96:33902) 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f518931e640 1 -- 172.21.15.96:0/2966496134 learned_addr learned my addr 172.21.15.96:0/2966496134 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:41.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f5188b1d640 1 -- 172.21.15.96:0/2966496134 >> v1:172.21.15.96:6789/0 conn(0x7f5184134e70 legacy=0x7f5184135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.293+0000 7f5188b1d640 1 -- 172.21.15.96:0/2966496134 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184136970 con 0x7f518409e820 2024-09-13T17:43:41.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f5188b1d640 1 --2- 172.21.15.96:0/2966496134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f518413ae50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5178009740 tx=0x7f517801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c654304c692b64a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f5183fff640 1 -- 172.21.15.96:0/2966496134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51780ab530 con 0x7f518409e820 2024-09-13T17:43:41.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f5183fff640 1 -- 172.21.15.96:0/2966496134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780a7050 con 0x7f518409e820 2024-09-13T17:43:41.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f5183fff640 1 -- 172.21.15.96:0/2966496134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f51780ab7e0 con 0x7f518409e820 2024-09-13T17:43:41.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- 172.21.15.96:0/2966496134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 msgr2=0x7f518413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 --2- 172.21.15.96:0/2966496134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f518413ae50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5178009740 tx=0x7f517801c420 comp rx=0 tx=0).stop 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- 172.21.15.96:0/2966496134 shutdown_connections 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 --2- 172.21.15.96:0/2966496134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f518413ae50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- 172.21.15.96:0/2966496134 >> 172.21.15.96:0/2966496134 conn(0x7f5184098a60 msgr2=0x7f518409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- 172.21.15.96:0/2966496134 shutdown_connections 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- 172.21.15.96:0/2966496134 wait complete. 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 Processor -- start 2024-09-13T17:43:41.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- start start 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518b582640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51840a36b0 con 0x7f518409e820 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50460/0 (socket says 172.21.15.96:50460) 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518931e640 1 -- 172.21.15.96:0/3838220352 learned_addr learned my addr 172.21.15.96:0/3838220352 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.297+0000 7f518931e640 1 -- 172.21.15.96:0/3838220352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009420 con 0x7f518409e820 2024-09-13T17:43:41.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f517c00ca90 tx=0x7f517c00cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 -- 172.21.15.96:0/3838220352 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c004ca0 con 0x7f518409e820 2024-09-13T17:43:41.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 -- 172.21.15.96:0/3838220352 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c004e00 con 0x7f518409e820 2024-09-13T17:43:41.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51840a33f0 con 0x7f518409e820 2024-09-13T17:43:41.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 -- 172.21.15.96:0/3838220352 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c00aba0 con 0x7f518409e820 2024-09-13T17:43:41.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840a1890 con 0x7f518409e820 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 -- 172.21.15.96:0/3838220352 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f517c00ad00 con 0x7f518409e820 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 0x7f5170037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5181ffb640 1 -- 172.21.15.96:0/3838220352 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f517c047740 con 0x7f518409e820 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5188b1d640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 0x7f5170037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f5188b1d640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 0x7f5170037a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51780ad000 tx=0x7f5178008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518e6cb000 1 Processor -- start 2024-09-13T17:43:41.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518e6cb000 1 -- start start 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518e6cb000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518e6cb000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd5cd2e80 con 0x558cd5e11710 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50476/0 (socket says 172.21.15.96:50476) 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 -- 172.21.15.96:0/9924868 learned_addr learned my addr 172.21.15.96:0/9924868 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 -- 172.21.15.96:0/9924868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd5955890 con 0x558cd5e11710 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.301+0000 7f518931e640 1 --2- 172.21.15.96:0/9924868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f517c012040 tx=0x7f517c037050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ea7cdb3dafe74d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f516affd640 1 -- 172.21.15.96:0/9924868 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c0059b0 con 0x558cd5e11710 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f516affd640 1 -- 172.21.15.96:0/9924868 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c045280 con 0x558cd5e11710 2024-09-13T17:43:41.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f516affd640 1 -- 172.21.15.96:0/9924868 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c0453f0 con 0x558cd5e11710 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/9924868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 msgr2=0x558cd5e078f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 --2- 172.21.15.96:0/9924868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f517c012040 tx=0x7f517c037050 comp rx=0 tx=0).stop 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/9924868 shutdown_connections 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 --2- 172.21.15.96:0/9924868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e078f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/9924868 >> 172.21.15.96:0/9924868 conn(0x558cd5e02230 msgr2=0x558cd5e071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/9924868 shutdown_connections 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/9924868 wait complete. 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- start start 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 Processor -- start 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd5c60af0 con 0x558cd5e11710 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 10 client.0 _ll_register_callbacks cb 0x558cd5df6ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50484/0 (socket says 172.21.15.96:50484) 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518931e640 1 -- 172.21.15.96:0/1227657425 learned_addr learned my addr 172.21.15.96:0/1227657425 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518931e640 1 -- 172.21.15.96:0/1227657425 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51840a1890 con 0x558cd5e11710 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518931e640 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f517c012040 tx=0x7f517c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835b655f3d4d3cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c03f070 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f517c011770 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd5c165f0 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c011930 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 10 client.4801 fetch_fsmap learned FSMap version 20 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x558cd5c1eab0 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cd5c165f0 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f517c024020 con 0x558cd5e11710 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 10 client.4801 fetch_fsmap finished waiting for FSMap version 20 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 10 client.4801 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 20 client.4801 populate_metadata read hostname 'smithi096' 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e17130 0x558cd5e3db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 4 client.4801 mds_command: new command op to 4743 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:43:41.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 --> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cd5c56790 con 0x558cd5e17130 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5188b1d640 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e17130 0x558cd5e3db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5188b1d640 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e17130 0x558cd5e3db20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9339ce5e235223bb server_cookie=fe665dbfeb9578e in_seq=0 out_seq=0 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 10 client.4801 ms_handle_connect on v2:172.21.15.96:6828/2227270685 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 1 -- 172.21.15.96:0/1227657425 <== mds.0 v2:172.21.15.96:6828/2227270685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x558cd5c56790 con 0x558cd5e17130 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f5163fff640 10 client.4801 handle_command_reply: tid=0 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 client.4801 shutdown 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 msgr2=0x558cd5e37760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.305+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f517c012040 tx=0x7f517c0017b0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 shutdown_connections 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e17130 0x558cd5e3db20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1227657425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e37760 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 >> 172.21.15.96:0/1227657425 conn(0x558cd5e02230 msgr2=0x558cd5e0a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f5163fff640 0 client.4801 ms_handle_reset on v2:172.21.15.96:6828/2227270685 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 shutdown_connections 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 1 -- 172.21.15.96:0/1227657425 wait complete. 2024-09-13T17:43:41.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.309+0000 7f518e6cb000 20 client.4801 trim_cache size 0 max 16384 2024-09-13T17:43:41.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518e6cb000 1 Processor -- start 2024-09-13T17:43:41.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518e6cb000 1 -- start start 2024-09-13T17:43:41.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518e6cb000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518e6cb000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd5c5e0c0 con 0x558cd5e11710 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50500/0 (socket says 172.21.15.96:50500) 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518931e640 1 -- 172.21.15.96:0/3401131622 learned_addr learned my addr 172.21.15.96:0/3401131622 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518931e640 1 -- 172.21.15.96:0/3401131622 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cd5c04c00 con 0x558cd5e11710 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f518931e640 1 --2- 172.21.15.96:0/3401131622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f517c00afd0 tx=0x7f517c03c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cf1ce9d3ace259 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f516b7fe640 1 -- 172.21.15.96:0/3401131622 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c010300 con 0x558cd5e11710 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f516b7fe640 1 -- 172.21.15.96:0/3401131622 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c045280 con 0x558cd5e11710 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.321+0000 7f516b7fe640 1 -- 172.21.15.96:0/3401131622 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c011770 con 0x558cd5e11710 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/3401131622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 msgr2=0x558cd5e0c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/3401131622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f517c00afd0 tx=0x7f517c03c050 comp rx=0 tx=0).stop 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/3401131622 shutdown_connections 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/3401131622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e0c890 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/3401131622 >> 172.21.15.96:0/3401131622 conn(0x558cd5e02230 msgr2=0x558cd5e0ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/3401131622 shutdown_connections 2024-09-13T17:43:41.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/3401131622 wait complete. 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- start start 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 Processor -- start 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cd5c599b0 con 0x558cd5e11710 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 10 client.0 _ll_register_callbacks cb 0x558cd5c08a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518931e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50512/0 (socket says 172.21.15.96:50512) 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518931e640 1 -- 172.21.15.96:0/1722732853 learned_addr learned my addr 172.21.15.96:0/1722732853 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518931e640 1 -- 172.21.15.96:0/1722732853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cd5c1eab0 con 0x558cd5e11710 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518931e640 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f517c010810 tx=0x7f517c010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c5e6f63e6aacb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:43:41.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f517c010300 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f517c047950 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cd5c29460 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f517c01b520 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 10 client.4803 fetch_fsmap learned FSMap version 20 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x558cd5955cc0 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558cd5c29460 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 20) v1 ==== 2024+0+0 (secure 0 0 0) 0x7f517c04e070 con 0x558cd5e11710 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 10 client.4803 fetch_fsmap finished waiting for FSMap version 20 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 10 client.4803 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 20 client.4803 populate_metadata read hostname 'smithi096' 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e73970 0x558cd5e73d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 4 client.4803 mds_command: new command op to 4743 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 --> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558cd5c229c0 con 0x558cd5e73970 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5188b1d640 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e73970 0x558cd5e73d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 10 client.4803 ms_handle_connect on v2:172.21.15.96:6828/2227270685 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5188b1d640 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e73970 0x558cd5e73d50 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a99a90de9ed276fa server_cookie=935f9b27860a72a1 in_seq=0 out_seq=0 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 1 -- 172.21.15.96:0/1722732853 <== mds.0 v2:172.21.15.96:6828/2227270685 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x558cd5c229c0 con 0x558cd5e73970 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 10 client.4803 handle_command_reply: tid=0 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 client.4803 shutdown 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 msgr2=0x558cd5e73430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f517c010810 tx=0x7f517c010610 comp rx=0 tx=0).stop 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 shutdown_connections 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:6828/2227270685,v1:172.21.15.96:6829/2227270685] conn(0x558cd5e73970 0x558cd5e73d50 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 --2- 172.21.15.96:0/1722732853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cd5e11710 0x558cd5e73430 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 >> 172.21.15.96:0/1722732853 conn(0x558cd5e02230 msgr2=0x558cd5e10750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f5163fff640 0 client.4803 ms_handle_reset on v2:172.21.15.96:6828/2227270685 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 shutdown_connections 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 1 -- 172.21.15.96:0/1722732853 wait complete. 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518e6cb000 20 client.4803 trim_cache size 0 max 16384 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "a54aa7b1-da4c-43a3-b94f-1607e77cbcc4", 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4743, 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-13T17:43:41.334 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 4.3873564600000003, 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 20, 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 37, 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 37, 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 8.8068167939999995 2024-09-13T17:43:41.335 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 msgr2=0x7f5170037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 0x7f5170037a30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f51780ad000 tx=0x7f5178008040 comp rx=0 tx=0).stop 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 msgr2=0x7f51841cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f517c00ca90 tx=0x7f517c00cf50 comp rx=0 tx=0).stop 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 shutdown_connections 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f5170035570 0x7f5170037a30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 --2- 172.21.15.96:0/3838220352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f518409e820 0x7f51841cda00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.325+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 >> 172.21.15.96:0/3838220352 conn(0x7f5184098a60 msgr2=0x7f5184099610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.329+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 shutdown_connections 2024-09-13T17:43:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.329+0000 7f518b582640 1 -- 172.21.15.96:0/3838220352 wait complete. 2024-09-13T17:43:41.345 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:43:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.449+0000 7f3ba7aea640 1 Processor -- start 2024-09-13T17:43:41.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.449+0000 7f3ba7aea640 1 -- start start 2024-09-13T17:43:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3ba009e820 0x7f3ba013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba00a68b0 con 0x7f3ba0134e70 2024-09-13T17:43:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba009ddc0 con 0x7f3ba009e820 2024-09-13T17:43:41.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5085640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3ba009e820 0x7f3ba013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5886640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3ba0134e70 0x7f3ba0135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:33916/0 (socket says 172.21.15.96:33916) 2024-09-13T17:43:41.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5886640 1 -- 172.21.15.96:0/2169900767 learned_addr learned my addr 172.21.15.96:0/2169900767 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:41.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5085640 1 -- 172.21.15.96:0/2169900767 >> v1:172.21.15.96:6789/0 conn(0x7f3ba0134e70 legacy=0x7f3ba0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5085640 1 -- 172.21.15.96:0/2169900767 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba0136970 con 0x7f3ba009e820 2024-09-13T17:43:41.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba5085640 1 --2- 172.21.15.96:0/2169900767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba013ae50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009740 tx=0x7f3b9c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d2e435c5bf069a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba4884640 1 -- 172.21.15.96:0/2169900767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c0ab530 con 0x7f3ba009e820 2024-09-13T17:43:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba4884640 1 -- 172.21.15.96:0/2169900767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c0a7050 con 0x7f3ba009e820 2024-09-13T17:43:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba4884640 1 -- 172.21.15.96:0/2169900767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b9c0ab7e0 con 0x7f3ba009e820 2024-09-13T17:43:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- 172.21.15.96:0/2169900767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 msgr2=0x7f3ba013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/2169900767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba013ae50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009740 tx=0x7f3b9c01c420 comp rx=0 tx=0).stop 2024-09-13T17:43:41.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- 172.21.15.96:0/2169900767 shutdown_connections 2024-09-13T17:43:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/2169900767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba013ae50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- 172.21.15.96:0/2169900767 >> 172.21.15.96:0/2169900767 conn(0x7f3ba0098a60 msgr2=0x7f3ba009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- 172.21.15.96:0/2169900767 shutdown_connections 2024-09-13T17:43:41.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.453+0000 7f3ba7aea640 1 -- 172.21.15.96:0/2169900767 wait complete. 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 Processor -- start 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 -- start start 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba00a44c0 con 0x7f3ba009e820 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5886640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50538/0 (socket says 172.21.15.96:50538) 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5886640 1 -- 172.21.15.96:0/3210134528 learned_addr learned my addr 172.21.15.96:0/3210134528 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5886640 1 -- 172.21.15.96:0/3210134528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009420 con 0x7f3ba009e820 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5886640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c480 tx=0x7f3b9000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b90005c20 con 0x7f3ba009e820 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b90005d80 con 0x7f3ba009e820 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba00a1960 con 0x7f3ba009e820 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b900071a0 con 0x7f3ba009e820 2024-09-13T17:43:41.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba00a41f0 con 0x7f3ba009e820 2024-09-13T17:43:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f3b90007300 con 0x7f3ba009e820 2024-09-13T17:43:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 0x7f3b84037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:41.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba00008e0 con 0x7f3ba009e820 2024-09-13T17:43:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f3b90046da0 con 0x7f3ba009e820 2024-09-13T17:43:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5085640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 0x7f3b84037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:41.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.457+0000 7f3ba5085640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 0x7f3b84037a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ad000 tx=0x7f3b9c008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:41.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.461+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3b9001b3b0 con 0x7f3ba009e820 2024-09-13T17:43:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.637+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3ba00a1680 con 0x7f3ba009e820 2024-09-13T17:43:41.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3b967fc640 1 -- 172.21.15.96:0/3210134528 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3b90017070 con 0x7f3ba009e820 2024-09-13T17:43:41.643 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 msgr2=0x7f3b84037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 0x7f3b84037a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0ad000 tx=0x7f3b9c008040 comp rx=0 tx=0).stop 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 msgr2=0x7f3ba01cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000c480 tx=0x7f3b9000c940 comp rx=0 tx=0).stop 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 shutdown_connections 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f3b84035570 0x7f3b84037a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 --2- 172.21.15.96:0/3210134528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ba009e820 0x7f3ba01cdae0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 >> 172.21.15.96:0/3210134528 conn(0x7f3ba0098a60 msgr2=0x7f3ba0098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:41.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 shutdown_connections 2024-09-13T17:43:41.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:41.641+0000 7f3ba7aea640 1 -- 172.21.15.96:0/3210134528 wait complete. 2024-09-13T17:43:41.659 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:43:41.659 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:43:41.659 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:43:41.659 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:43:41.660 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:41.660 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:43:41.660 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:43:41.660 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:43:41.660 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:41.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.660 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.666 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42461sec preferred_lft 42461sec 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.667 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:43:41.668 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:41.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.669 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:43:41.718 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:43:41.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.720 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:43:41.770 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:43:41.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.771 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:43:41.771 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:43:41.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:43:41.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:43:41.772 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:43:41.772 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:43:41.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.845 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:43:41.891 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:43:41.891 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:43:41.892 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:41.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:41.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-13T17:43:41.946 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:43:41.946 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:41.946 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:41.999 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:43:41.999 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:41.999 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:42.046 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:42.104 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:42.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:42.157 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:42.158 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:42.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.159 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:42.207 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:43:42.207 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:43:42.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:42.267 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:42.267 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:42.320 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:42.321 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:42.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:42.321 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:42.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-13T17:43:42.329+0000 7fa5fd06f580 -1 init, newargv = 0x556242a473e0 newargc=15 2024-09-13T17:43:42.332 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38269]: starting ceph client 2024-09-13T17:43:42.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38269]: starting fuse 2024-09-13T17:43:42.880 INFO:tasks.ceph.mds.b.smithi096.stderr:2024-09-13T17:43:42.877+0000 7f52d91f0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:43:43.438 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:43.455 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:43.455 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:43.513 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:43.514 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:43.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:43.515 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:43.563 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:43.563 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T17:43:43.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:43.563 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:43.563 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:43.563 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:43.563 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:43.564 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:43:43.565 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:43.683 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.38269.asok 2024-09-13T17:43:43.865 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38269.asok 2024-09-13T17:43:43.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:43.868 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38269.asok status 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "38269", 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:44.023 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4809, 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4809 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3935638152 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:44.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3935638152 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4809 192.168.0.1:0/3935638152", 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/3935638152", 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 20, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 37, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:43:44.025 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:44.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.037 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:43:44.037 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:44.042 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:43:44.042 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:44.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.043 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:43:44.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:43:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.261+0000 7fdd7b988640 1 Processor -- start 2024-09-13T17:43:44.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.261+0000 7fdd7b988640 1 -- start start 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd7b988640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd7b988640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7409ed60 con 0x7fdd74135250 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd7b988640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7409ddc0 con 0x7fdd74134e70 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd79724640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd79724640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41010/0 (socket says 172.21.15.96:41010) 2024-09-13T17:43:44.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd78f23640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdd74135250 0x7fdd74135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39440/0 (socket says 172.21.15.96:39440) 2024-09-13T17:43:44.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd79724640 1 -- 172.21.15.96:0/261867569 learned_addr learned my addr 172.21.15.96:0/261867569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:44.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd79724640 1 -- 172.21.15.96:0/261867569 >> v1:172.21.15.96:6789/0 conn(0x7fdd74135250 legacy=0x7fdd74135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd79724640 1 -- 172.21.15.96:0/261867569 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd740a68b0 con 0x7fdd74134e70 2024-09-13T17:43:44.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.265+0000 7fdd6bfff640 1 -- 172.21.15.96:0/261867569 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897085724 0 0) 0x7fdd70001120 con 0x7fdd74135250 2024-09-13T17:43:44.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 --2- 172.21.15.96:0/261867569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097f0 tx=0x7fdd6401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fa280926ccc878 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd6bfff640 1 -- 172.21.15.96:0/261867569 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd640ac540 con 0x7fdd74134e70 2024-09-13T17:43:44.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd6bfff640 1 -- 172.21.15.96:0/261867569 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640ac6a0 con 0x7fdd74134e70 2024-09-13T17:43:44.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd6bfff640 1 -- 172.21.15.96:0/261867569 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd640ac960 con 0x7fdd74134e70 2024-09-13T17:43:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/261867569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 msgr2=0x7fdd7409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 --2- 172.21.15.96:0/261867569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fdd640097f0 tx=0x7fdd6401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/261867569 shutdown_connections 2024-09-13T17:43:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 --2- 172.21.15.96:0/261867569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409e820 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/261867569 >> 172.21.15.96:0/261867569 conn(0x7fdd74098a60 msgr2=0x7fdd7409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/261867569 shutdown_connections 2024-09-13T17:43:44.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/261867569 wait complete. 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 Processor -- start 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- start start 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd740a1fa0 con 0x7fdd74134e70 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41022/0 (socket says 172.21.15.96:41022) 2024-09-13T17:43:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 -- 172.21.15.96:0/3962599710 learned_addr learned my addr 172.21.15.96:0/3962599710 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 -- 172.21.15.96:0/3962599710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640094d0 con 0x7fdd74134e70 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd79724640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdd6401a110 tx=0x7fdd6401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd64004160 con 0x7fdd74134e70 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd640042c0 con 0x7fdd74134e70 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdd64003620 con 0x7fdd74134e70 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd740a2b30 con 0x7fdd74134e70 2024-09-13T17:43:44.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.269+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd740a2240 con 0x7fdd74134e70 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdd640b3040 con 0x7fdd74134e70 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd740a2990 con 0x7fdd74134e70 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd69ffb640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fdd640e1e20 con 0x7fdd74134e70 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd78f23640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:44.276 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.273+0000 7fdd78f23640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000c10 tx=0x7fdd70001320 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:44.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.277+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdd740a2990 con 0x7fdd74134e70 2024-09-13T17:43:44.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdd740a2990 con 0x7fdd74134e70 2024-09-13T17:43:44.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd69ffb640 1 -- 172.21.15.96:0/3962599710 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdd740a2990 con 0x7fdd74134e70 2024-09-13T17:43:44.475 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 msgr2=0x7fdd58037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdd70000c10 tx=0x7fdd70001320 comp rx=0 tx=0).stop 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 msgr2=0x7fdd7409f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fdd6401a110 tx=0x7fdd6401ad20 comp rx=0 tx=0).stop 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 shutdown_connections 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fdd58035570 0x7fdd58037a30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 --2- 172.21.15.96:0/3962599710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd74134e70 0x7fdd7409f280 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 >> 172.21.15.96:0/3962599710 conn(0x7fdd74098a60 msgr2=0x7fdd74098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 shutdown_connections 2024-09-13T17:43:44.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:44.473+0000 7fdd7b988640 1 -- 172.21.15.96:0/3962599710 wait complete. 2024-09-13T17:43:44.493 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:43:44.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:43:44.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:43:44.493 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:43:44.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:44.494 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:43:44.494 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:43:44.494 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:43:44.494 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:44.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.494 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42458sec preferred_lft 42458sec 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:43:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.501 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:43:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:43:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:43:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:43:44.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.502 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:43:44.553 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:43:44.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.554 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:43:44.610 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:43:44.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:43:44.612 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:43:44.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.690 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:43:44.743 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:43:44.743 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:43:44.743 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:44.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:44.743 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:43:44.799 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:43:44.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:44.799 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:44.846 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:43:44.846 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:43:44.847 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:44.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:44.952 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:44.952 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:45.010 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:45.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:45.012 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:45.059 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:45.059 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:43:45.059 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:43:45.104 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:45.117 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:45.117 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:45.169 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:45.170 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:45.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:45.170 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:45.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:43:45.177+0000 7fba8165b580 -1 init, newargv = 0x55d89b98a7d0 newargc=15 2024-09-13T17:43:45.181 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[38376]: starting ceph client 2024-09-13T17:43:45.185 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:45.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[38376]: starting fuse 2024-09-13T17:43:46.287 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:43:46.348 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:43:46.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:43:46.402 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:43:46.404 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:43:46.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:46.404 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:43:46.450 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:43:46.451 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:43:46.451 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:46.451 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:43:46.452 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:43:46.453 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:43:46.453 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:43:46.453 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:43:46.453 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:43:46.453 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:43:46.571 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.38376.asok 2024-09-13T17:43:46.754 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.38376.asok 2024-09-13T17:43:46.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:46.755 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.38376.asok status 2024-09-13T17:43:46.915 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:43:46.915 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "38376", 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4815, 2024-09-13T17:43:46.916 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4815 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 705277613 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:43:46.917 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 705277613 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4815 192.168.0.2:0/705277613", 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/705277613", 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 20, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 37, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:43:46.918 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:43:46.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:46.932 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:43:46.932 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:46.939 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:43:46.939 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:46.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:46.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:46.996 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-09-13T17:43:47.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe10b84b640 1 Processor -- start 2024-09-13T17:43:47.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe10b84b640 1 -- start start 2024-09-13T17:43:47.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe10b84b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe10409dc20 0x7fe1041351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe10b84b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe10406e130 con 0x7fe10406f6d0 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe10b84b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1040a6750 con 0x7fe10409dc20 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe1095e7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe10406f6d0 0x7fe104134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39452/0 (socket says 172.21.15.96:39452) 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe108de6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe10409dc20 0x7fe1041351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe1095e7640 1 -- 172.21.15.96:0/2835491267 learned_addr learned my addr 172.21.15.96:0/2835491267 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe108de6640 1 -- 172.21.15.96:0/2835491267 >> v1:172.21.15.96:6789/0 conn(0x7fe10406f6d0 legacy=0x7fe104134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe108de6640 1 -- 172.21.15.96:0/2835491267 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1040712e0 con 0x7fe10409dc20 2024-09-13T17:43:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.157+0000 7fe103fff640 1 -- 172.21.15.96:0/2835491267 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916524809 0 0) 0x7fe10406e130 con 0x7fe10406f6d0 2024-09-13T17:43:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe108de6640 1 --2- 172.21.15.96:0/2835491267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10409dc20 0x7fe1041351c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009740 tx=0x7fe0f801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1380c15806284118 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe103fff640 1 -- 172.21.15.96:0/2835491267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0f80ab400 con 0x7fe10409dc20 2024-09-13T17:43:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe103fff640 1 -- 172.21.15.96:0/2835491267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80a7050 con 0x7fe10409dc20 2024-09-13T17:43:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe103fff640 1 -- 172.21.15.96:0/2835491267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0f80ab6b0 con 0x7fe10409dc20 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- 172.21.15.96:0/2835491267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10409dc20 msgr2=0x7fe1041351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2835491267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10409dc20 0x7fe1041351c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8009740 tx=0x7fe0f801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- 172.21.15.96:0/2835491267 shutdown_connections 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2835491267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10409dc20 0x7fe1041351c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- 172.21.15.96:0/2835491267 >> 172.21.15.96:0/2835491267 conn(0x7fe104098a60 msgr2=0x7fe10409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- 172.21.15.96:0/2835491267 shutdown_connections 2024-09-13T17:43:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- 172.21.15.96:0/2835491267 wait complete. 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 Processor -- start 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 -- start start 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.161+0000 7fe10b84b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe10b84b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1040a3d70 con 0x7fe10406f6d0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe1095e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe1095e7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41044/0 (socket says 172.21.15.96:41044) 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe1095e7640 1 -- 172.21.15.96:0/2348274949 learned_addr learned my addr 172.21.15.96:0/2348274949 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe1095e7640 1 -- 172.21.15.96:0/2348274949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f8009420 con 0x7fe10406f6d0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe1095e7640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c840 tx=0x7fe0fc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0fc00a660 con 0x7fe10406f6d0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0fc00a7e0 con 0x7fe10406f6d0 2024-09-13T17:43:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0fc00aac0 con 0x7fe10406f6d0 2024-09-13T17:43:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1040a3630 con 0x7fe10406f6d0 2024-09-13T17:43:47.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1041393c0 con 0x7fe10406f6d0 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1040a4110 con 0x7fe10406f6d0 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fe0fc00ac20 con 0x7fe10406f6d0 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 0x7fe0f0037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe0fc047160 con 0x7fe10406f6d0 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe108de6640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 0x7fe0f0037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:47.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.165+0000 7fe108de6640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 0x7fe0f0037a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ad000 tx=0x7fe0f8008040 comp rx=0 tx=0).ready entity=mgr.4630 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:47.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.169+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe0fc01a6b0 con 0x7fe10406f6d0 2024-09-13T17:43:47.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:47.365+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) v1 -- 0x7fe1040a3830 con 0x7fe10406f6d0 2024-09-13T17:43:48.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.121+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fe0fc01c3b0 con 0x7fe10406f6d0 2024-09-13T17:43:48.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe101ffb640 1 -- 172.21.15.96:0/2348274949 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v16) v1 ==== 84+0+0 (secure 0 0 0) 0x7fe0fc007970 con 0x7fe10406f6d0 2024-09-13T17:43:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 msgr2=0x7fe0f0037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 0x7fe0f0037a30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ad000 tx=0x7fe0f8008040 comp rx=0 tx=0).stop 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 msgr2=0x7fe1040a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00c840 tx=0x7fe0fc00cd00 comp rx=0 tx=0).stop 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 shutdown_connections 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fe0f0035570 0x7fe0f0037a30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 --2- 172.21.15.96:0/2348274949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe10406f6d0 0x7fe1040a25b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 >> 172.21.15.96:0/2348274949 conn(0x7fe104098a60 msgr2=0x7fe104131c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 shutdown_connections 2024-09-13T17:43:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.125+0000 7fe10b84b640 1 -- 172.21.15.96:0/2348274949 wait complete. 2024-09-13T17:43:48.149 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:48.149 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:48.155 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:48.248 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:48.245+0000 7f994de2b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:43:48.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.305+0000 7fc9948cc640 1 Processor -- start 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.305+0000 7fc9948cc640 1 -- start start 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.305+0000 7fc9948cc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc990136100 0x7fc99009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99009d090 con 0x7fc9901364e0 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99009cd80 con 0x7fc990136100 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98e59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc990136100 0x7fc99009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98dd9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc9901364e0 0x7fc990099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39480/0 (socket says 172.21.15.96:39480) 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98dd9b640 1 -- 172.21.15.96:0/1739829668 learned_addr learned my addr 172.21.15.96:0/1739829668 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98e59c640 1 -- 172.21.15.96:0/1739829668 >> v1:172.21.15.96:6789/0 conn(0x7fc9901364e0 legacy=0x7fc990099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98d59a640 1 -- 172.21.15.96:0/1739829668 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569369875 0 0) 0x7fc99009d090 con 0x7fc9901364e0 2024-09-13T17:43:48.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98e59c640 1 -- 172.21.15.96:0/1739829668 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99009dec0 con 0x7fc990136100 2024-09-13T17:43:48.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98e59c640 1 --2- 172.21.15.96:0/1739829668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc99009d7e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc978009740 tx=0x7fc97801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79725e8da173c3de server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98d59a640 1 -- 172.21.15.96:0/1739829668 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9780ac540 con 0x7fc990136100 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98d59a640 1 -- 172.21.15.96:0/1739829668 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9780ac6a0 con 0x7fc990136100 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc98d59a640 1 -- 172.21.15.96:0/1739829668 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9780ac960 con 0x7fc990136100 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- 172.21.15.96:0/1739829668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 msgr2=0x7fc99009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 --2- 172.21.15.96:0/1739829668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc99009d7e0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc978009740 tx=0x7fc97801c350 comp rx=0 tx=0).stop 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- 172.21.15.96:0/1739829668 shutdown_connections 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 --2- 172.21.15.96:0/1739829668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc99009d7e0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- 172.21.15.96:0/1739829668 >> 172.21.15.96:0/1739829668 conn(0x7fc99006f270 msgr2=0x7fc990071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- 172.21.15.96:0/1739829668 shutdown_connections 2024-09-13T17:43:48.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.309+0000 7fc9948cc640 1 -- 172.21.15.96:0/1739829668 wait complete. 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 Processor -- start 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 -- start start 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9900a4f10 con 0x7fc990136100 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41094/0 (socket says 172.21.15.96:41094) 2024-09-13T17:43:48.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98e59c640 1 -- 172.21.15.96:0/3417694735 learned_addr learned my addr 172.21.15.96:0/3417694735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98e59c640 1 -- 172.21.15.96:0/3417694735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978009420 con 0x7fc990136100 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98e59c640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc978019f80 tx=0x7fc97801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9780b8580 con 0x7fc990136100 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9780b86e0 con 0x7fc990136100 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9900a60a0 con 0x7fc990136100 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9780b89a0 con 0x7fc990136100 2024-09-13T17:43:48.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc990130e10 con 0x7fc990136100 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fc9780b8b00 con 0x7fc990136100 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc990134390 con 0x7fc990136100 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 0x7fc970037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fc9780e2440 con 0x7fc990136100 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98dd9b640 1 -- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 msgr2=0x7fc970037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:48.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.313+0000 7fc98dd9b640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 0x7fc970037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:48.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.317+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc9780c88b0 con 0x7fc990136100 2024-09-13T17:43:48.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.477+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc99009d7e0 con 0x7fc990136100 2024-09-13T17:43:48.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc986ffd640 1 -- 172.21.15.96:0/3417694735 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+10491 (secure 0 0 0) 0x7fc9780b1090 con 0x7fc990136100 2024-09-13T17:43:48.484 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:48.484 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":37,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:37.942012+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:48.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 msgr2=0x7fc970037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:48.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 0x7fc970037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 msgr2=0x7fc9900a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc978019f80 tx=0x7fc97801ab90 comp rx=0 tx=0).stop 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 shutdown_connections 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fc9700355c0 0x7fc970037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 --2- 172.21.15.96:0/3417694735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc990136100 0x7fc9900a5b60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 >> 172.21.15.96:0/3417694735 conn(0x7fc99006f270 msgr2=0x7fc990070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 shutdown_connections 2024-09-13T17:43:48.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.481+0000 7fc9948cc640 1 -- 172.21.15.96:0/3417694735 wait complete. 2024-09-13T17:43:48.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:48.503 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:48.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.609+0000 7f9280ff0640 1 Processor -- start 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.609+0000 7f9280ff0640 1 -- start start 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9280ff0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f927c13af50 0x7f927c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9280ff0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927c138050 con 0x7f927c09d690 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9280ff0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927c09c410 con 0x7f927c13af50 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9279d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f927c13af50 0x7f927c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f927a59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f927c09d690 0x7f927c138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39488/0 (socket says 172.21.15.96:39488) 2024-09-13T17:43:48.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f927a59c640 1 -- 172.21.15.96:0/1100432319 learned_addr learned my addr 172.21.15.96:0/1100432319 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:48.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9279d9b640 1 -- 172.21.15.96:0/1100432319 >> v1:172.21.15.96:6789/0 conn(0x7f927c09d690 legacy=0x7f927c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9279d9b640 1 -- 172.21.15.96:0/1100432319 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c09cba0 con 0x7f927c13af50 2024-09-13T17:43:48.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9279d9b640 1 --2- 172.21.15.96:0/1100432319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c13af50 0x7f927c13d340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f926c009740 tx=0x7f926c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c024b0b4512ff65 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f927959a640 1 -- 172.21.15.96:0/1100432319 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f926c0a8840 con 0x7f927c13af50 2024-09-13T17:43:48.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f927959a640 1 -- 172.21.15.96:0/1100432319 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c0a89a0 con 0x7f927c13af50 2024-09-13T17:43:48.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f927959a640 1 -- 172.21.15.96:0/1100432319 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f926c0a8c60 con 0x7f927c13af50 2024-09-13T17:43:48.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9280ff0640 1 -- 172.21.15.96:0/1100432319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c13af50 msgr2=0x7f927c13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.613+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1100432319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c13af50 0x7f927c13d340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f926c009740 tx=0x7f926c01ec10 comp rx=0 tx=0).stop 2024-09-13T17:43:48.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1100432319 shutdown_connections 2024-09-13T17:43:48.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1100432319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c13af50 0x7f927c13d340 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1100432319 >> 172.21.15.96:0/1100432319 conn(0x7f927c06bc60 msgr2=0x7f927c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1100432319 shutdown_connections 2024-09-13T17:43:48.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1100432319 wait complete. 2024-09-13T17:43:48.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 Processor -- start 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- start start 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c0a4f90 con 0x7f927c09d690 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f927a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f927a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41100/0 (socket says 172.21.15.96:41100) 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f927a59c640 1 -- 172.21.15.96:0/1374741811 learned_addr learned my addr 172.21.15.96:0/1374741811 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f927a59c640 1 -- 172.21.15.96:0/1374741811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c009420 con 0x7f927c09d690 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f927a59c640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f927000c380 tx=0x7f927000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9270005c20 con 0x7f927c09d690 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270005d80 con 0x7f927c09d690 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c0a2c20 con 0x7f927c09d690 2024-09-13T17:43:48.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9270007200 con 0x7f927c09d690 2024-09-13T17:43:48.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c1373b0 con 0x7f927c09d690 2024-09-13T17:43:48.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927c0a3f70 con 0x7f927c09d690 2024-09-13T17:43:48.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f9270007360 con 0x7f927c09d690 2024-09-13T17:43:48.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 0x7f925c037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9270046a80 con 0x7f927c09d690 2024-09-13T17:43:48.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9279d9b640 1 -- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 msgr2=0x7f925c037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:48.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.617+0000 7f9279d9b640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 0x7f925c037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:48.623 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:48.621+0000 7f994de2b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:43:48.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.625+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f927c0a3f70 con 0x7f927c09d690 2024-09-13T17:43:48.758 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:48.757+0000 7f994de2b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:43:48.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f927c138050 con 0x7f927c09d690 2024-09-13T17:43:48.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f926affd640 1 -- 172.21.15.96:0/1374741811 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f927c138050 con 0x7f927c09d690 2024-09-13T17:43:48.791 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:48.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 msgr2=0x7f925c037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:48.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 0x7f925c037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 msgr2=0x7f927c1c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f927000c380 tx=0x7f927000c840 comp rx=0 tx=0).stop 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 shutdown_connections 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f925c0355c0 0x7f925c037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 --2- 172.21.15.96:0/1374741811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f927c09d690 0x7f927c1c9820 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 >> 172.21.15.96:0/1374741811 conn(0x7f927c06bc60 msgr2=0x7f927c131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 shutdown_connections 2024-09-13T17:43:48.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.789+0000 7f9280ff0640 1 -- 172.21.15.96:0/1374741811 wait complete. 2024-09-13T17:43:48.809 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:48.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c9a03640 1 Processor -- start 2024-09-13T17:43:48.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c9a03640 1 -- start start 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c9a03640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65c4137ed0 0x7f65c409f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c9a03640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c406b020 con 0x7f65c40a10d0 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c9a03640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c40a0430 con 0x7f65c4137ed0 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65c4137ed0 0x7f65c409f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f65c4137ed0 0x7f65c409f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41104/0 (socket says 172.21.15.96:41104) 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c27fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f65c40a10d0 0x7f65c41382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39496/0 (socket says 172.21.15.96:39496) 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c2ffd640 1 -- 172.21.15.96:0/1770018585 learned_addr learned my addr 172.21.15.96:0/1770018585 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c2ffd640 1 -- 172.21.15.96:0/1770018585 >> v1:172.21.15.96:6789/0 conn(0x7f65c40a10d0 legacy=0x7f65c41382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c1ffb640 1 -- 172.21.15.96:0/1770018585 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108790105 0 0) 0x7f65c406b020 con 0x7f65c40a10d0 2024-09-13T17:43:48.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.917+0000 7f65c2ffd640 1 -- 172.21.15.96:0/1770018585 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c406cb30 con 0x7f65c4137ed0 2024-09-13T17:43:48.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c2ffd640 1 --2- 172.21.15.96:0/1770018585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c4137ed0 0x7f65c409f620 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097f0 tx=0x7f65ac01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3614ddfe51fe90 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c1ffb640 1 -- 172.21.15.96:0/1770018585 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0ab530 con 0x7f65c4137ed0 2024-09-13T17:43:48.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c1ffb640 1 -- 172.21.15.96:0/1770018585 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac0a7050 con 0x7f65c4137ed0 2024-09-13T17:43:48.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c1ffb640 1 -- 172.21.15.96:0/1770018585 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0ab7e0 con 0x7f65c4137ed0 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- 172.21.15.96:0/1770018585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c4137ed0 msgr2=0x7f65c409f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 --2- 172.21.15.96:0/1770018585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c4137ed0 0x7f65c409f620 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0097f0 tx=0x7f65ac01c450 comp rx=0 tx=0).stop 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- 172.21.15.96:0/1770018585 shutdown_connections 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 --2- 172.21.15.96:0/1770018585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c4137ed0 0x7f65c409f620 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- 172.21.15.96:0/1770018585 >> 172.21.15.96:0/1770018585 conn(0x7f65c406f270 msgr2=0x7f65c4071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- 172.21.15.96:0/1770018585 shutdown_connections 2024-09-13T17:43:48.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- 172.21.15.96:0/1770018585 wait complete. 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 Processor -- start 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- start start 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.921+0000 7f65c9a03640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c411f820 con 0x7f65c40a10d0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41114/0 (socket says 172.21.15.96:41114) 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c2ffd640 1 -- 172.21.15.96:0/2025733873 learned_addr learned my addr 172.21.15.96:0/2025733873 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c2ffd640 1 -- 172.21.15.96:0/2025733873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0094d0 con 0x7f65c40a10d0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c2ffd640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f65ac01a080 tx=0x7f65ac01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0abea0 con 0x7f65c40a10d0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ac004020 con 0x7f65c40a10d0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c41207a0 con 0x7f65c40a10d0 2024-09-13T17:43:48.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65ac0042c0 con 0x7f65c40a10d0 2024-09-13T17:43:48.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c411eee0 con 0x7f65c40a10d0 2024-09-13T17:43:48.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c40a78f0 con 0x7f65c40a10d0 2024-09-13T17:43:48.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f65ac0b2040 con 0x7f65c40a10d0 2024-09-13T17:43:48.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 0x7f65a4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:48.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f65ac0e1920 con 0x7f65c40a10d0 2024-09-13T17:43:48.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c27fc640 1 -- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 msgr2=0x7f65a4037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:48.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.925+0000 7f65c27fc640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 0x7f65a4037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:48.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:48.929+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65ac0c6300 con 0x7f65c40a10d0 2024-09-13T17:43:49.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.077+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65c411ebd0 con 0x7f65c40a10d0 2024-09-13T17:43:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.077+0000 7f65bbfff640 1 -- 172.21.15.96:0/2025733873 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f65ac0b0020 con 0x7f65c40a10d0 2024-09-13T17:43:49.081 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 msgr2=0x7f65a4037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 0x7f65a4037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 msgr2=0x7f65c4120260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f65ac01a080 tx=0x7f65ac01ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 shutdown_connections 2024-09-13T17:43:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f65a40355c0 0x7f65a4037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 --2- 172.21.15.96:0/2025733873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65c40a10d0 0x7f65c4120260 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 >> 172.21.15.96:0/2025733873 conn(0x7f65c406f270 msgr2=0x7f65c413c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 shutdown_connections 2024-09-13T17:43:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.081+0000 7f65c9a03640 1 -- 172.21.15.96:0/2025733873 wait complete. 2024-09-13T17:43:49.098 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:49.191 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.189+0000 7f994de2b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805f47b640 1 Processor -- start 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805f47b640 1 -- start start 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805f47b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f805809b0a0 0x7f80581350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805f47b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8058135a30 con 0x7f80581355e0 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805f47b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8058134af0 con 0x7f805809b0a0 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805d217640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f805809b0a0 0x7f80581350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805ca16640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f80581355e0 0x7f8058131540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39498/0 (socket says 172.21.15.96:39498) 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805d217640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f805809b0a0 0x7f80581350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41126/0 (socket says 172.21.15.96:41126) 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805ca16640 1 -- 172.21.15.96:0/1571210408 learned_addr learned my addr 172.21.15.96:0/1571210408 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:49.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805d217640 1 -- 172.21.15.96:0/1571210408 >> v1:172.21.15.96:6789/0 conn(0x7f80581355e0 legacy=0x7f8058131540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.221+0000 7f805d217640 1 -- 172.21.15.96:0/1571210408 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805809b480 con 0x7f805809b0a0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805d217640 1 --2- 172.21.15.96:0/1571210408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f80581350a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8050009d90 tx=0x7f805001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9224cfb094edbbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f804ffff640 1 -- 172.21.15.96:0/1571210408 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8058132460 con 0x7f805809b0a0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f804ffff640 1 -- 172.21.15.96:0/1571210408 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500a5810 con 0x7f805809b0a0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f804ffff640 1 -- 172.21.15.96:0/1571210408 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80500a5ad0 con 0x7f805809b0a0 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- 172.21.15.96:0/1571210408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 msgr2=0x7f80581350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 --2- 172.21.15.96:0/1571210408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f80581350a0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8050009d90 tx=0x7f805001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- 172.21.15.96:0/1571210408 shutdown_connections 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 --2- 172.21.15.96:0/1571210408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f80581350a0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- 172.21.15.96:0/1571210408 >> 172.21.15.96:0/1571210408 conn(0x7f805806e020 msgr2=0x7f8058070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- 172.21.15.96:0/1571210408 shutdown_connections 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- 172.21.15.96:0/1571210408 wait complete. 2024-09-13T17:43:49.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 Processor -- start 2024-09-13T17:43:49.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.225+0000 7f805f47b640 1 -- start start 2024-09-13T17:43:49.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805f47b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805f47b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805806a600 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805d217640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805d217640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41128/0 (socket says 172.21.15.96:41128) 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805d217640 1 -- 172.21.15.96:0/657588802 learned_addr learned my addr 172.21.15.96:0/657588802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805d217640 1 -- 172.21.15.96:0/657588802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8050009a70 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805d217640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f805001a510 tx=0x7f8050019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80500ad780 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80500ad8e0 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8058136a70 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80500adbc0 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80581304e0 con 0x7f805809b0a0 2024-09-13T17:43:49.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f80500b4050 con 0x7f805809b0a0 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80580a82b0 con 0x7f805809b0a0 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 0x7f803c037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f80500e1f10 con 0x7f805809b0a0 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805ca16640 1 -- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 msgr2=0x7f803c037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:49.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.229+0000 7f805ca16640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 0x7f803c037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:49.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.237+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f80500ace00 con 0x7f805809b0a0 2024-09-13T17:43:49.247 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.245+0000 7f994de2b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:43:49.296 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.293+0000 7f994de2b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:43:49.372 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.369+0000 7f994de2b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:43:49.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.385+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8058130100 con 0x7f805809b0a0 2024-09-13T17:43:49.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.385+0000 7f804dffb640 1 -- 172.21.15.96:0/657588802 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80500ac3a0 con 0x7f805809b0a0 2024-09-13T17:43:49.389 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:49.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 msgr2=0x7f803c037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:49.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 0x7f803c037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 msgr2=0x7f8058135c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f805001a510 tx=0x7f8050019680 comp rx=0 tx=0).stop 2024-09-13T17:43:49.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 shutdown_connections 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f803c0355c0 0x7f803c037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 --2- 172.21.15.96:0/657588802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f805809b0a0 0x7f8058135c20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 >> 172.21.15.96:0/657588802 conn(0x7f805806e020 msgr2=0x7f805812b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 shutdown_connections 2024-09-13T17:43:49.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.389+0000 7f805f47b640 1 -- 172.21.15.96:0/657588802 wait complete. 2024-09-13T17:43:49.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:49.424 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.421+0000 7f994de2b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:43:49.499 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.497+0000 7f994de2b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe3ccf640 1 Processor -- start 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe3ccf640 1 -- start start 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe3ccf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6fdc0715c0 0x7f6fdc13ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe3ccf640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0706a0 con 0x7f6fdc12f530 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe3ccf640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fdc070920 con 0x7f6fdc0715c0 2024-09-13T17:43:49.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe1a6b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6fdc12f530 0x7f6fdc12f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39512/0 (socket says 172.21.15.96:39512) 2024-09-13T17:43:49.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe1a6b640 1 -- 172.21.15.96:0/1708830631 learned_addr learned my addr 172.21.15.96:0/1708830631 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:49.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe126a640 1 --2- 172.21.15.96:0/1708830631 >> v2:172.21.15.96:3300/0 conn(0x7f6fdc0715c0 0x7f6fdc13ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe0a69640 1 -- 172.21.15.96:0/1708830631 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383678279 0 0) 0x7f6fdc0706a0 con 0x7f6fdc12f530 2024-09-13T17:43:49.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe0a69640 1 -- 172.21.15.96:0/1708830631 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc8003530 con 0x7f6fdc12f530 2024-09-13T17:43:49.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe126a640 1 -- 172.21.15.96:0/1708830631 >> v1:172.21.15.96:6789/0 conn(0x7f6fdc12f530 legacy=0x7f6fdc12f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.529+0000 7f6fe126a640 1 -- 172.21.15.96:0/1708830631 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc130020 con 0x7f6fdc0715c0 2024-09-13T17:43:49.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe126a640 1 --2- 172.21.15.96:0/1708830631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc13ae70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009740 tx=0x7f6fcc01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6401f889eaf52cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe0a69640 1 -- 172.21.15.96:0/1708830631 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc004160 con 0x7f6fdc0715c0 2024-09-13T17:43:49.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe0a69640 1 -- 172.21.15.96:0/1708830631 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0a7050 con 0x7f6fdc0715c0 2024-09-13T17:43:49.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe0a69640 1 -- 172.21.15.96:0/1708830631 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fcc004450 con 0x7f6fdc0715c0 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/1708830631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 msgr2=0x7f6fdc13ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/1708830631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc13ae70 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009740 tx=0x7f6fcc01c370 comp rx=0 tx=0).stop 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/1708830631 shutdown_connections 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/1708830631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc13ae70 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/1708830631 >> 172.21.15.96:0/1708830631 conn(0x7f6fdc0a6b40 msgr2=0x7f6fdc0a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/1708830631 shutdown_connections 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/1708830631 wait complete. 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 Processor -- start 2024-09-13T17:43:49.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- start start 2024-09-13T17:43:49.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc09c560 con 0x7f6fdc0715c0 2024-09-13T17:43:49.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe1a6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe1a6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41146/0 (socket says 172.21.15.96:41146) 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe1a6b640 1 -- 172.21.15.96:0/3678744800 learned_addr learned my addr 172.21.15.96:0/3678744800 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe1a6b640 1 -- 172.21.15.96:0/3678744800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc009420 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe1a6b640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400cb40 tx=0x7f6fd400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fd400abf0 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd400ad50 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc09c270 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6fd4012400 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc09b890 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc09b2c0 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.533+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f6fd4007070 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.537+0000 7f6fd27fc640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 0x7f6fc0037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.537+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f6fd4047780 con 0x7f6fdc0715c0 2024-09-13T17:43:49.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.537+0000 7f6fe126a640 1 -- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 msgr2=0x7f6fc0037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:49.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.537+0000 7f6fe126a640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 0x7f6fc0037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:49.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.541+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6fd401b3b0 con 0x7f6fdc0715c0 2024-09-13T17:43:49.551 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.549+0000 7f994de2b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:43:49.692 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.689+0000 7f994de2b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:43:49.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6fdc066860 con 0x7f6fdc0715c0 2024-09-13T17:43:49.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fd27fc640 1 -- 172.21.15.96:0/3678744800 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6fd4017020 con 0x7f6fdc0715c0 2024-09-13T17:43:49.703 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 msgr2=0x7f6fc0037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 0x7f6fc0037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 msgr2=0x7f6fdc09a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400cb40 tx=0x7f6fd400a060 comp rx=0 tx=0).stop 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 shutdown_connections 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f6fc00355c0 0x7f6fc0037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 --2- 172.21.15.96:0/3678744800 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6fdc0715c0 0x7f6fdc09a490 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 >> 172.21.15.96:0/3678744800 conn(0x7f6fdc0a6b40 msgr2=0x7f6fdc0a5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 shutdown_connections 2024-09-13T17:43:49.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.701+0000 7f6fe3ccf640 1 -- 172.21.15.96:0/3678744800 wait complete. 2024-09-13T17:43:49.721 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:43:49.721 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:43:49.746 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.741+0000 7f994de2b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:43:49.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:49.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8857af5640 1 Processor -- start 2024-09-13T17:43:49.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8857af5640 1 -- start start 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8857af5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f885009a810 0x7f885013a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8857af5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f885009c020 con 0x7f8850098a60 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8857af5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885009cc90 con 0x7f885009a810 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855090640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f885009a810 0x7f885013a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855891640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8850098a60 0x7f8850098e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39528/0 (socket says 172.21.15.96:39528) 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855090640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f885009a810 0x7f885013a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41150/0 (socket says 172.21.15.96:41150) 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855891640 1 -- 172.21.15.96:0/3742423638 learned_addr learned my addr 172.21.15.96:0/3742423638 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855090640 1 -- 172.21.15.96:0/3742423638 >> v1:172.21.15.96:6789/0 conn(0x7f8850098a60 legacy=0x7f8850098e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f885488f640 1 -- 172.21.15.96:0/3742423638 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716109547 0 0) 0x7f885009c020 con 0x7f8850098a60 2024-09-13T17:43:49.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.949+0000 7f8855090640 1 -- 172.21.15.96:0/3742423638 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885009c2e0 con 0x7f885009a810 2024-09-13T17:43:49.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855090640 1 --2- 172.21.15.96:0/3742423638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f885009a810 0x7f885013a110 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f884c009740 tx=0x7f884c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2531699d098bebf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f885488f640 1 -- 172.21.15.96:0/3742423638 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f884c0ab530 con 0x7f885009a810 2024-09-13T17:43:49.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f885488f640 1 -- 172.21.15.96:0/3742423638 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c0a7050 con 0x7f885009a810 2024-09-13T17:43:49.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f885488f640 1 -- 172.21.15.96:0/3742423638 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f884c0ab7e0 con 0x7f885009a810 2024-09-13T17:43:49.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- 172.21.15.96:0/3742423638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f885009a810 msgr2=0x7f885013a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:49.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 --2- 172.21.15.96:0/3742423638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f885009a810 0x7f885013a110 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f884c009740 tx=0x7f884c01c390 comp rx=0 tx=0).stop 2024-09-13T17:43:49.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- 172.21.15.96:0/3742423638 shutdown_connections 2024-09-13T17:43:49.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 --2- 172.21.15.96:0/3742423638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f885009a810 0x7f885013a110 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:49.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- 172.21.15.96:0/3742423638 >> 172.21.15.96:0/3742423638 conn(0x7f885006e350 msgr2=0x7f8850070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:49.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- 172.21.15.96:0/3742423638 shutdown_connections 2024-09-13T17:43:49.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- 172.21.15.96:0/3742423638 wait complete. 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 Processor -- start 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- start start 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8857af5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88501c95b0 con 0x7f8850098a60 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855891640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855891640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41154/0 (socket says 172.21.15.96:41154) 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855891640 1 -- 172.21.15.96:0/1104842605 learned_addr learned my addr 172.21.15.96:0/1104842605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855891640 1 -- 172.21.15.96:0/1104842605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884c009420 con 0x7f8850098a60 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f8855891640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f884000e6e0 tx=0x7f884000eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.953+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8840007200 con 0x7f8850098a60 2024-09-13T17:43:49.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8840007380 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f8857af5640 1 -- 172.21.15.96:0/1104842605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88500a2430 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8840007660 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f8857af5640 1 -- 172.21.15.96:0/1104842605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8850139c80 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f88400016f0 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c005260 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f88467fc640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 0x7f8834037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f884004c030 con 0x7f8850098a60 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f8855090640 1 -- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 msgr2=0x7f8834037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:49.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.957+0000 7f8855090640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 0x7f8834037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:49.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:49.961+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8840022710 con 0x7f8850098a60 2024-09-13T17:43:49.969 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:43:49.969 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:43:49.970 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:43:49.977 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:49.973+0000 7f994de2b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:43:50.115 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:50.113+0000 7f994de2b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:50.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.117+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f882c0057e0 con 0x7f8850098a60 2024-09-13T17:43:50.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.121+0000 7f88467fc640 1 -- 172.21.15.96:0/1104842605 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+10491 (secure 0 0 0) 0x7f884004a030 con 0x7f8850098a60 2024-09-13T17:43:50.125 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:50.125 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":37,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:37.942012+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 msgr2=0x7f8834037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 0x7f8834037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 msgr2=0x7f88501c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f884000e6e0 tx=0x7f884000eba0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 shutdown_connections 2024-09-13T17:43:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f88340355c0 0x7f8834037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 --2- 172.21.15.96:0/1104842605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8850098a60 0x7f88501c97a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 >> 172.21.15.96:0/1104842605 conn(0x7f885006e350 msgr2=0x7f8850139e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 shutdown_connections 2024-09-13T17:43:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.125+0000 7f8833fff640 1 -- 172.21.15.96:0/1104842605 wait complete. 2024-09-13T17:43:50.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:50.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:50.233 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:50.229+0000 7f994de2b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:43:50.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.253+0000 7f9b91e72640 1 Processor -- start 2024-09-13T17:43:50.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b91e72640 1 -- start start 2024-09-13T17:43:50.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b91e72640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b8c13af50 0x7f9b8c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b91e72640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b8c138050 con 0x7f9b8c09d690 2024-09-13T17:43:50.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b91e72640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b8c09c410 con 0x7f9b8c13af50 2024-09-13T17:43:50.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9b8c13af50 0x7f9b8c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9b8c09d690 0x7f9b8c138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39536/0 (socket says 172.21.15.96:39536) 2024-09-13T17:43:50.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8b7fe640 1 -- 172.21.15.96:0/1436389900 learned_addr learned my addr 172.21.15.96:0/1436389900 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:50.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8affd640 1 -- 172.21.15.96:0/1436389900 >> v1:172.21.15.96:6789/0 conn(0x7f9b8c09d690 legacy=0x7f9b8c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8affd640 1 -- 172.21.15.96:0/1436389900 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c09cba0 con 0x7f9b8c13af50 2024-09-13T17:43:50.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.257+0000 7f9b8a7fc640 1 -- 172.21.15.96:0/1436389900 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866158982 0 0) 0x7f9b7c001120 con 0x7f9b8c09d690 2024-09-13T17:43:50.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8affd640 1 --2- 172.21.15.96:0/1436389900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c13af50 0x7f9b8c13d340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009740 tx=0x7f9b8001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa675160a084a360 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8a7fc640 1 -- 172.21.15.96:0/1436389900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b800a8840 con 0x7f9b8c13af50 2024-09-13T17:43:50.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8a7fc640 1 -- 172.21.15.96:0/1436389900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b800a89a0 con 0x7f9b8c13af50 2024-09-13T17:43:50.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8a7fc640 1 -- 172.21.15.96:0/1436389900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b800a8c60 con 0x7f9b8c13af50 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/1436389900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c13af50 msgr2=0x7f9b8c13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 --2- 172.21.15.96:0/1436389900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c13af50 0x7f9b8c13d340 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9b80009740 tx=0x7f9b8001ec10 comp rx=0 tx=0).stop 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/1436389900 shutdown_connections 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 --2- 172.21.15.96:0/1436389900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c13af50 0x7f9b8c13d340 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/1436389900 >> 172.21.15.96:0/1436389900 conn(0x7f9b8c06bc60 msgr2=0x7f9b8c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/1436389900 shutdown_connections 2024-09-13T17:43:50.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/1436389900 wait complete. 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 Processor -- start 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- start start 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c0a5c90 con 0x7f9b8c09d690 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41168/0 (socket says 172.21.15.96:41168) 2024-09-13T17:43:50.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8b7fe640 1 -- 172.21.15.96:0/3979527128 learned_addr learned my addr 172.21.15.96:0/3979527128 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8b7fe640 1 -- 172.21.15.96:0/3979527128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b80009420 con 0x7f9b8c09d690 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b8b7fe640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c390 tx=0x7f9b7c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c007410 con 0x7f9b8c09d690 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7c007590 con 0x7f9b8c09d690 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c0a6f20 con 0x7f9b8c09d690 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b7c0047f0 con 0x7f9b8c09d690 2024-09-13T17:43:50.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.261+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c137400 con 0x7f9b8c09d690 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c09bc00 con 0x7f9b8c09d690 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f9b7c004950 con 0x7f9b8c09d690 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b73fff640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 0x7f9b78037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b8affd640 1 -- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 msgr2=0x7f9b78037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9b7c047460 con 0x7f9b8c09d690 2024-09-13T17:43:50.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.265+0000 7f9b8affd640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 0x7f9b78037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:50.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.269+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9b8c09bc00 con 0x7f9b8c09d690 2024-09-13T17:43:50.299 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:50.297+0000 7f994de2b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:43:50.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.421+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b8c138050 con 0x7f9b8c09d690 2024-09-13T17:43:50.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.421+0000 7f9b73fff640 1 -- 172.21.15.96:0/3979527128 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b8c138050 con 0x7f9b8c09d690 2024-09-13T17:43:50.425 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 msgr2=0x7f9b78037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 0x7f9b78037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 msgr2=0x7f9b8c0a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00c390 tx=0x7f9b7c00c850 comp rx=0 tx=0).stop 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 shutdown_connections 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9b780355c0 0x7f9b78037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 --2- 172.21.15.96:0/3979527128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9b8c09d690 0x7f9b8c0a69e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 >> 172.21.15.96:0/3979527128 conn(0x7f9b8c06bc60 msgr2=0x7f9b8c09f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 shutdown_connections 2024-09-13T17:43:50.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.425+0000 7f9b91e72640 1 -- 172.21.15.96:0/3979527128 wait complete. 2024-09-13T17:43:50.437 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:50.433+0000 7f994de2b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:43:50.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:50.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.549+0000 7fea52ca6640 1 Processor -- start 2024-09-13T17:43:50.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.549+0000 7fea52ca6640 1 -- start start 2024-09-13T17:43:50.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.549+0000 7fea52ca6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fea4c132a10 0x7fea4c132df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4c09d040 con 0x7fea4c09c180 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea4c09f080 con 0x7fea4c132a10 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea50a42640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fea4c132a10 0x7fea4c132df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fea4c09c180 0x7fea4c069f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39550/0 (socket says 172.21.15.96:39550) 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4bfff640 1 -- 172.21.15.96:0/990385452 learned_addr learned my addr 172.21.15.96:0/990385452 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea50a42640 1 -- 172.21.15.96:0/990385452 >> v1:172.21.15.96:6789/0 conn(0x7fea4c09c180 legacy=0x7fea4c069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4b7fe640 1 -- 172.21.15.96:0/990385452 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666181430 0 0) 0x7fea4c09d040 con 0x7fea4c09c180 2024-09-13T17:43:50.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea50a42640 1 -- 172.21.15.96:0/990385452 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c132810 con 0x7fea4c132a10 2024-09-13T17:43:50.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea50a42640 1 --2- 172.21.15.96:0/990385452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c132a10 0x7fea4c132df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fea40009740 tx=0x7fea4001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e59c69b169610c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4b7fe640 1 -- 172.21.15.96:0/990385452 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea400a4ea0 con 0x7fea4c132a10 2024-09-13T17:43:50.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4b7fe640 1 -- 172.21.15.96:0/990385452 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea400a7050 con 0x7fea4c132a10 2024-09-13T17:43:50.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea4b7fe640 1 -- 172.21.15.96:0/990385452 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea400ab680 con 0x7fea4c132a10 2024-09-13T17:43:50.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- 172.21.15.96:0/990385452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c132a10 msgr2=0x7fea4c132df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 --2- 172.21.15.96:0/990385452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c132a10 0x7fea4c132df0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fea40009740 tx=0x7fea4001c330 comp rx=0 tx=0).stop 2024-09-13T17:43:50.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- 172.21.15.96:0/990385452 shutdown_connections 2024-09-13T17:43:50.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 --2- 172.21.15.96:0/990385452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c132a10 0x7fea4c132df0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- 172.21.15.96:0/990385452 >> 172.21.15.96:0/990385452 conn(0x7fea4c06e020 msgr2=0x7fea4c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- 172.21.15.96:0/990385452 shutdown_connections 2024-09-13T17:43:50.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.553+0000 7fea52ca6640 1 -- 172.21.15.96:0/990385452 wait complete. 2024-09-13T17:43:50.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 Processor -- start 2024-09-13T17:43:50.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 -- start start 2024-09-13T17:43:50.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c13af70 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea50a42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea50a42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41180/0 (socket says 172.21.15.96:41180) 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea50a42640 1 -- 172.21.15.96:0/620445930 learned_addr learned my addr 172.21.15.96:0/620445930 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea50a42640 1 -- 172.21.15.96:0/620445930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea40009420 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea50a42640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea40018fa0 tx=0x7fea4001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea400abe80 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea400b7450 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea400b7730 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c0a5100 con 0x7fea4c09c180 2024-09-13T17:43:50.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c09a8b0 con 0x7fea4c09c180 2024-09-13T17:43:50.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fea400b7890 con 0x7fea4c09c180 2024-09-13T17:43:50.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c0a5320 con 0x7fea4c09c180 2024-09-13T17:43:50.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 0x7fea30037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea4bfff640 1 -- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 msgr2=0x7fea30037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:50.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fea400e2310 con 0x7fea4c09c180 2024-09-13T17:43:50.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.557+0000 7fea4bfff640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 0x7fea30037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:50.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.561+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fea400c74e0 con 0x7fea4c09c180 2024-09-13T17:43:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea4c073a50 con 0x7fea4c09c180 2024-09-13T17:43:50.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea497fa640 1 -- 172.21.15.96:0/620445930 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea400b00a0 con 0x7fea4c09c180 2024-09-13T17:43:50.715 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:50.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 msgr2=0x7fea30037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 0x7fea30037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 msgr2=0x7fea4c0a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fea40018fa0 tx=0x7fea4001abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 shutdown_connections 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7fea300355c0 0x7fea30037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 --2- 172.21.15.96:0/620445930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea4c09c180 0x7fea4c0a3e80 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 >> 172.21.15.96:0/620445930 conn(0x7fea4c06e020 msgr2=0x7fea4c0710e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 shutdown_connections 2024-09-13T17:43:50.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.713+0000 7fea52ca6640 1 -- 172.21.15.96:0/620445930 wait complete. 2024-09-13T17:43:50.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:50.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed2de8640 1 Processor -- start 2024-09-13T17:43:50.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed2de8640 1 -- start start 2024-09-13T17:43:50.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed2de8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9ecc0a6510 0x7f9ecc13af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed2de8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ecc070600 con 0x7f9ecc0a4900 2024-09-13T17:43:50.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed2de8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ecc070830 con 0x7f9ecc0a6510 2024-09-13T17:43:50.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed0b84640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9ecc0a4900 0x7f9ecc0a4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39562/0 (socket says 172.21.15.96:39562) 2024-09-13T17:43:50.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ecbfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9ecc0a6510 0x7f9ecc13af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ed0b84640 1 -- 172.21.15.96:0/3067228905 learned_addr learned my addr 172.21.15.96:0/3067228905 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:50.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ecbfff640 1 -- 172.21.15.96:0/3067228905 >> v1:172.21.15.96:6789/0 conn(0x7f9ecc0a4900 legacy=0x7f9ecc0a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ecbfff640 1 -- 172.21.15.96:0/3067228905 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc0a5c70 con 0x7f9ecc0a6510 2024-09-13T17:43:50.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.841+0000 7f9ecb7fe640 1 -- 172.21.15.96:0/3067228905 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631048862 0 0) 0x7f9ecc070600 con 0x7f9ecc0a4900 2024-09-13T17:43:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ecbfff640 1 --2- 172.21.15.96:0/3067228905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a6510 0x7f9ecc13af50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009740 tx=0x7f9eb801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1997c738125b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ecb7fe640 1 -- 172.21.15.96:0/3067228905 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9eb80ab530 con 0x7f9ecc0a6510 2024-09-13T17:43:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ecb7fe640 1 -- 172.21.15.96:0/3067228905 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb80a7050 con 0x7f9ecc0a6510 2024-09-13T17:43:50.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ecb7fe640 1 -- 172.21.15.96:0/3067228905 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9eb80ab7e0 con 0x7f9ecc0a6510 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 -- 172.21.15.96:0/3067228905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a6510 msgr2=0x7f9ecc13af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/3067228905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a6510 0x7f9ecc13af50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f9eb8009740 tx=0x7f9eb801c420 comp rx=0 tx=0).stop 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 -- 172.21.15.96:0/3067228905 shutdown_connections 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/3067228905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a6510 0x7f9ecc13af50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 -- 172.21.15.96:0/3067228905 >> 172.21.15.96:0/3067228905 conn(0x7f9ecc06a150 msgr2=0x7f9ecc06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:50.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 -- 172.21.15.96:0/3067228905 shutdown_connections 2024-09-13T17:43:50.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.845+0000 7f9ed2de8640 1 -- 172.21.15.96:0/3067228905 wait complete. 2024-09-13T17:43:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 Processor -- start 2024-09-13T17:43:50.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 -- start start 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc133f40 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed0b84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed0b84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41188/0 (socket says 172.21.15.96:41188) 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed0b84640 1 -- 172.21.15.96:0/1973019524 learned_addr learned my addr 172.21.15.96:0/1973019524 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed0b84640 1 -- 172.21.15.96:0/1973019524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8009420 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed0b84640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c420 tx=0x7f9ec000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec00074d0 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec0007630 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc09a590 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec0004960 con 0x7f9ecc0a4900 2024-09-13T17:43:50.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc09b530 con 0x7f9ecc0a4900 2024-09-13T17:43:50.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f9ec0004ac0 con 0x7f9ecc0a4900 2024-09-13T17:43:50.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc098a60 con 0x7f9ecc0a4900 2024-09-13T17:43:50.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 0x7f9eb0037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:50.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9ec0046ef0 con 0x7f9ecc0a4900 2024-09-13T17:43:50.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ecbfff640 1 -- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 msgr2=0x7f9eb0037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:50.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.849+0000 7f9ecbfff640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 0x7f9eb0037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:50.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:50.853+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9ec001acd0 con 0x7f9ecc0a4900 2024-09-13T17:43:50.931 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:50.928+0000 7f994de2b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:43:51.005 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.000+0000 7f994de2b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:43:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.000+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9ecc0a4d60 con 0x7f9ecc0a4900 2024-09-13T17:43:51.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ec97fa640 1 -- 172.21.15.96:0/1973019524 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9ec0018020 con 0x7f9ecc0a4900 2024-09-13T17:43:51.006 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 msgr2=0x7f9eb0037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 0x7f9eb0037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 msgr2=0x7f9ecc09aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000c420 tx=0x7f9ec000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 shutdown_connections 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f9eb00355c0 0x7f9eb0037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 --2- 172.21.15.96:0/1973019524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ecc0a4900 0x7f9ecc09aff0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 >> 172.21.15.96:0/1973019524 conn(0x7f9ecc06a150 msgr2=0x7f9ecc134960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 shutdown_connections 2024-09-13T17:43:51.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.004+0000 7f9ed2de8640 1 -- 172.21.15.96:0/1973019524 wait complete. 2024-09-13T17:43:51.024 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:43:51.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.144+0000 7f7b0a1fc640 1 Processor -- start 2024-09-13T17:43:51.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.144+0000 7f7b0a1fc640 1 -- start start 2024-09-13T17:43:51.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b0a1fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7b04137110 0x7f7b041374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b0a1fc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0409a180 con 0x7f7b0409ba80 2024-09-13T17:43:51.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b0a1fc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0409adc0 con 0x7f7b04137110 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7b04137110 0x7f7b041374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7b04137110 0x7f7b041374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41204/0 (socket says 172.21.15.96:41204) 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 -- 172.21.15.96:0/3936066032 learned_addr learned my addr 172.21.15.96:0/3936066032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7afaffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7b0409ba80 0x7f7b04069f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39566/0 (socket says 172.21.15.96:39566) 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 -- 172.21.15.96:0/3936066032 >> v1:172.21.15.96:6789/0 conn(0x7f7b0409ba80 legacy=0x7f7b04069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 -- 172.21.15.96:0/3936066032 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04099120 con 0x7f7b04137110 2024-09-13T17:43:51.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b037fe640 1 --2- 172.21.15.96:0/3936066032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04137110 0x7f7b041374f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097f0 tx=0x7f7af401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f61cc819dee129c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.153 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f994de2b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:43:51.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b02ffd640 1 -- 172.21.15.96:0/3936066032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af40a8050 con 0x7f7b04137110 2024-09-13T17:43:51.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b02ffd640 1 -- 172.21.15.96:0/3936066032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40ac520 con 0x7f7b04137110 2024-09-13T17:43:51.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.148+0000 7f7b02ffd640 1 -- 172.21.15.96:0/3936066032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af40ac7e0 con 0x7f7b04137110 2024-09-13T17:43:51.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/3936066032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04137110 msgr2=0x7f7b041374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/3936066032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04137110 0x7f7b041374f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097f0 tx=0x7f7af401c450 comp rx=0 tx=0).stop 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/3936066032 shutdown_connections 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/3936066032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b04137110 0x7f7b041374f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/3936066032 >> 172.21.15.96:0/3936066032 conn(0x7f7b0406e020 msgr2=0x7f7b04070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/3936066032 shutdown_connections 2024-09-13T17:43:51.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/3936066032 wait complete. 2024-09-13T17:43:51.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 Processor -- start 2024-09-13T17:43:51.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- start start 2024-09-13T17:43:51.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040a5120 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b037fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41218/0 (socket says 172.21.15.96:41218) 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b037fe640 1 -- 172.21.15.96:0/2722718664 learned_addr learned my addr 172.21.15.96:0/2722718664 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b037fe640 1 -- 172.21.15.96:0/2722718664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40094d0 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b037fe640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7af40af040 tx=0x7f7af401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af40b7820 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40b79a0 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040a5430 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7af40b7c80 con 0x7f7b0409ba80 2024-09-13T17:43:51.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040a56f0 con 0x7f7b0409ba80 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b040a5260 con 0x7f7b0409ba80 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.152+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f7af40b3040 con 0x7f7b0409ba80 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.156+0000 7f7b00ff9640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 0x7f7ae4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.156+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f7af40e2390 con 0x7f7b0409ba80 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.156+0000 7f7afaffd640 1 -- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 msgr2=0x7f7ae4037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/526856805 2024-09-13T17:43:51.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.156+0000 7f7afaffd640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 0x7f7ae4037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:43:51.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.160+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7af40b4980 con 0x7f7b0409ba80 2024-09-13T17:43:51.230 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.228+0000 7f994de2b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:43:51.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.316+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b0409cf80 con 0x7f7b0409ba80 2024-09-13T17:43:51.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b00ff9640 1 -- 172.21.15.96:0/2722718664 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7af40b1070 con 0x7f7b0409ba80 2024-09-13T17:43:51.323 INFO:teuthology.orchestra.run.smithi096.stderr:listed 0 entries 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 msgr2=0x7f7ae4037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 0x7f7ae4037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 msgr2=0x7f7b040a5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f7af40af040 tx=0x7f7af401ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 shutdown_connections 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:6824/526856805,v1:172.21.15.96:6825/526856805] conn(0x7f7ae40355c0 0x7f7ae4037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 --2- 172.21.15.96:0/2722718664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b0409ba80 0x7f7b040a5b80 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 >> 172.21.15.96:0/2722718664 conn(0x7f7b0406e020 msgr2=0x7f7b04070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 shutdown_connections 2024-09-13T17:43:51.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.320+0000 7f7b0a1fc640 1 -- 172.21.15.96:0/2722718664 wait complete. 2024-09-13T17:43:51.340 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:43:51.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:51.340 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:43:51.350 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[38376]: fuse finished with error 0 and tester_r 0 2024-09-13T17:43:51.352 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:43:51.377 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:43:51.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:43:51.377 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:43:51.427 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-13T17:43:51.518 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.512+0000 7f994de2b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:43:51.573 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.568+0000 7f994de2b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:43:51.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f378e96e640 1 Processor -- start 2024-09-13T17:43:51.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f378e96e640 1 -- start start 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f378e96e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f378809afb0 0x7f3788139c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f378e96e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378809ca70 con 0x7f378809c690 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f378e96e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3788131910 con 0x7f378809afb0 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f3787fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f378809c690 0x7f3788131b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:39568/0 (socket says 172.21.15.96:39568) 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f3787fff640 1 -- 172.21.15.96:0/3260414353 learned_addr learned my addr 172.21.15.96:0/3260414353 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f37877fe640 1 --2- 172.21.15.96:0/3260414353 >> v2:172.21.15.96:3300/0 conn(0x7f378809afb0 0x7f3788139c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f37877fe640 1 -- 172.21.15.96:0/3260414353 >> v1:172.21.15.96:6789/0 conn(0x7f378809c690 legacy=0x7f3788131b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.588+0000 7f37877fe640 1 -- 172.21.15.96:0/3260414353 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378809aaf0 con 0x7f378809afb0 2024-09-13T17:43:51.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f37877fe640 1 --2- 172.21.15.96:0/3260414353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f3788139c60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f377c009740 tx=0x7f377c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9517a56cb27efcfc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f3786ffd640 1 -- 172.21.15.96:0/3260414353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f377c0ac570 con 0x7f378809afb0 2024-09-13T17:43:51.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f3786ffd640 1 -- 172.21.15.96:0/3260414353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f377c0ac6d0 con 0x7f378809afb0 2024-09-13T17:43:51.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f3786ffd640 1 -- 172.21.15.96:0/3260414353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f377c0ac990 con 0x7f378809afb0 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 -- 172.21.15.96:0/3260414353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 msgr2=0x7f3788139c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 --2- 172.21.15.96:0/3260414353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f3788139c60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f377c009740 tx=0x7f377c01c420 comp rx=0 tx=0).stop 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 -- 172.21.15.96:0/3260414353 shutdown_connections 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 --2- 172.21.15.96:0/3260414353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f3788139c60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 -- 172.21.15.96:0/3260414353 >> 172.21.15.96:0/3260414353 conn(0x7f378806df50 msgr2=0x7f3788070370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 -- 172.21.15.96:0/3260414353 shutdown_connections 2024-09-13T17:43:51.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.592+0000 7f378e96e640 1 -- 172.21.15.96:0/3260414353 wait complete. 2024-09-13T17:43:51.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 Processor -- start 2024-09-13T17:43:51.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 -- start start 2024-09-13T17:43:51.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:51.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37880a2cb0 con 0x7f378809afb0 2024-09-13T17:43:51.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3787fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:51.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3787fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41254/0 (socket says 172.21.15.96:41254) 2024-09-13T17:43:51.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3787fff640 1 -- 172.21.15.96:0/3051702764 learned_addr learned my addr 172.21.15.96:0/3051702764 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:51.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3787fff640 1 -- 172.21.15.96:0/3051702764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c009420 con 0x7f378809afb0 2024-09-13T17:43:51.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3787fff640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f377800c3a0 tx=0x7f377800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3778005c20 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3778005d80 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37880a3070 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3778007200 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37880a4010 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378809cd30 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) v1 ==== 43027+0+0 (secure 0 0 0) 0x7f3778007360 con 0x7f378809afb0 2024-09-13T17:43:51.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f37780468a0 con 0x7f378809afb0 2024-09-13T17:43:51.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.604+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3778012400 con 0x7f378809afb0 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d4aa0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d4aa0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d4aa0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d4aa0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d4aa0640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d7aa6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d7aa6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d7aa6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d7aa6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.655 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:43:51.652+0000 7f98d7aa6640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:43:51.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:51.804+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f37880a4260 con 0x7f378809afb0 2024-09-13T17:43:52.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.596+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7f3778012780 con 0x7f378809afb0 2024-09-13T17:43:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.600+0000 7f3784ff9640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 0x7f377403bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.600+0000 7f37877fe640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 0x7f377403bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.600+0000 7f37877fe640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 0x7f377403bd00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f377c0ae000 tx=0x7f377c0a7000 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f3784ff9640 1 -- 172.21.15.96:0/3051702764 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v21) v1 ==== 136+0+0 (secure 0 0 0) 0x7f377801c440 con 0x7f378809afb0 2024-09-13T17:43:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 msgr2=0x7f377403bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 0x7f377403bd00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f377c0ae000 tx=0x7f377c0a7000 comp rx=0 tx=0).stop 2024-09-13T17:43:52.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 msgr2=0x7f37880a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f377800c3a0 tx=0x7f377800c860 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 shutdown_connections 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f3774039910 0x7f377403bd00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 --2- 172.21.15.96:0/3051702764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f378809afb0 0x7f37880a3ad0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 >> 172.21.15.96:0/3051702764 conn(0x7f378806df50 msgr2=0x7f3788069f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.608+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 shutdown_connections 2024-09-13T17:43:52.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.612+0000 7f378e96e640 1 -- 172.21.15.96:0/3051702764 wait complete. 2024-09-13T17:43:52.633 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:52.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.784+0000 7fee69053640 1 Processor -- start 2024-09-13T17:43:52.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.784+0000 7fee69053640 1 -- start start 2024-09-13T17:43:52.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fee64137490 0x7fee64139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee64071670 con 0x7fee64134670 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee64071360 con 0x7fee64137490 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fee64137490 0x7fee64139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fee64137490 0x7fee64139880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48674/0 (socket says 172.21.15.96:48674) 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 -- 172.21.15.96:0/4150040464 learned_addr learned my addr 172.21.15.96:0/4150040464 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 -- 172.21.15.96:0/4150040464 >> v1:172.21.15.96:6789/0 conn(0x7fee64134670 legacy=0x7fee64134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 -- 172.21.15.96:0/4150040464 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640a7200 con 0x7fee64137490 2024-09-13T17:43:52.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee5a59c640 1 --2- 172.21.15.96:0/4150040464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64137490 0x7fee64139880 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00a990 tx=0x7fee4c01d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23d11dea5d1cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee6259c640 1 -- 172.21.15.96:0/4150040464 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c0a75b0 con 0x7fee64137490 2024-09-13T17:43:52.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee6259c640 1 -- 172.21.15.96:0/4150040464 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c0a7710 con 0x7fee64137490 2024-09-13T17:43:52.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee6259c640 1 -- 172.21.15.96:0/4150040464 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee4c0a79d0 con 0x7fee64137490 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- 172.21.15.96:0/4150040464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64137490 msgr2=0x7fee64139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 --2- 172.21.15.96:0/4150040464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64137490 0x7fee64139880 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fee4c00a990 tx=0x7fee4c01d820 comp rx=0 tx=0).stop 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- 172.21.15.96:0/4150040464 shutdown_connections 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 --2- 172.21.15.96:0/4150040464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64137490 0x7fee64139880 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- 172.21.15.96:0/4150040464 >> 172.21.15.96:0/4150040464 conn(0x7fee64098a60 msgr2=0x7fee6409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- 172.21.15.96:0/4150040464 shutdown_connections 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.788+0000 7fee69053640 1 -- 172.21.15.96:0/4150040464 wait complete. 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 Processor -- start 2024-09-13T17:43:52.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 -- start start 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee64130780 con 0x7fee64134670 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee62d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee62d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48686/0 (socket says 172.21.15.96:48686) 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee62d9d640 1 -- 172.21.15.96:0/3412491885 learned_addr learned my addr 172.21.15.96:0/3412491885 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee62d9d640 1 -- 172.21.15.96:0/3412491885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c00a670 con 0x7fee64134670 2024-09-13T17:43:52.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee62d9d640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c480 tx=0x7fee5400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee54005150 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee540052b0 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee54007030 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee64131a70 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee641301b0 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee64131720 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7fee54007190 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 0x7fee440377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7fee54046a60 con 0x7fee64134670 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5a59c640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 0x7fee440377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:52.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.792+0000 7fee5a59c640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 0x7fee440377b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0b0000 tx=0x7fee4c008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:52.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.796+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee540108d0 con 0x7fee64134670 2024-09-13T17:43:52.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.948+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee6409ae60 con 0x7fee64134670 2024-09-13T17:43:52.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.948+0000 7fee5bfff640 1 -- 172.21.15.96:0/3412491885 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+10989 (secure 0 0 0) 0x7fee540102e0 con 0x7fee64134670 2024-09-13T17:43:52.953 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:52.953 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":38,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:51.591671+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:52.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 msgr2=0x7fee440377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 0x7fee440377b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0b0000 tx=0x7fee4c008040 comp rx=0 tx=0).stop 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 msgr2=0x7fee6412ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fee5400c480 tx=0x7fee5400c940 comp rx=0 tx=0).stop 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 shutdown_connections 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fee440352f0 0x7fee440377b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 --2- 172.21.15.96:0/3412491885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee64134670 0x7fee6412ec60 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 >> 172.21.15.96:0/3412491885 conn(0x7fee64098a60 msgr2=0x7fee6412e030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 shutdown_connections 2024-09-13T17:43:52.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:52.952+0000 7fee69053640 1 -- 172.21.15.96:0/3412491885 wait complete. 2024-09-13T17:43:52.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:43:52.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:43:53.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff60405c640 1 Processor -- start 2024-09-13T17:43:53.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff60405c640 1 -- start start 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff60405c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff5fc09dc20 0x7ff5fc1351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff60405c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5fc06e130 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff60405c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5fc0a6750 con 0x7ff5fc09dc20 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff601df8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff5fc06f6d0 0x7ff5fc134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57596/0 (socket says 172.21.15.96:57596) 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.080+0000 7ff601df8640 1 -- 172.21.15.96:0/779177856 learned_addr learned my addr 172.21.15.96:0/779177856 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff6015f7640 1 --2- 172.21.15.96:0/779177856 >> v2:172.21.15.96:3300/0 conn(0x7ff5fc09dc20 0x7ff5fc1351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff600df6640 1 -- 172.21.15.96:0/779177856 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792455213 0 0) 0x7ff5fc06e130 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff600df6640 1 -- 172.21.15.96:0/779177856 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e8003530 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff6015f7640 1 -- 172.21.15.96:0/779177856 >> v1:172.21.15.96:6789/0 conn(0x7ff5fc06f6d0 legacy=0x7ff5fc134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff6015f7640 1 -- 172.21.15.96:0/779177856 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc0712e0 con 0x7ff5fc09dc20 2024-09-13T17:43:53.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff6015f7640 1 --2- 172.21.15.96:0/779177856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc09dc20 0x7ff5fc1351c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009740 tx=0x7ff5ec01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc62f9625a53c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff600df6640 1 -- 172.21.15.96:0/779177856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0ab400 con 0x7ff5fc09dc20 2024-09-13T17:43:53.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff600df6640 1 -- 172.21.15.96:0/779177856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec0a7050 con 0x7ff5fc09dc20 2024-09-13T17:43:53.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff600df6640 1 -- 172.21.15.96:0/779177856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec0ab690 con 0x7ff5fc09dc20 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 -- 172.21.15.96:0/779177856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc09dc20 msgr2=0x7ff5fc1351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 --2- 172.21.15.96:0/779177856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc09dc20 0x7ff5fc1351c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009740 tx=0x7ff5ec01c390 comp rx=0 tx=0).stop 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 -- 172.21.15.96:0/779177856 shutdown_connections 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 --2- 172.21.15.96:0/779177856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc09dc20 0x7ff5fc1351c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 -- 172.21.15.96:0/779177856 >> 172.21.15.96:0/779177856 conn(0x7ff5fc098a60 msgr2=0x7ff5fc09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 -- 172.21.15.96:0/779177856 shutdown_connections 2024-09-13T17:43:53.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.084+0000 7ff60405c640 1 -- 172.21.15.96:0/779177856 wait complete. 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 Processor -- start 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 -- start start 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5fc11e260 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff601df8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff601df8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48706/0 (socket says 172.21.15.96:48706) 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff601df8640 1 -- 172.21.15.96:0/1282592293 learned_addr learned my addr 172.21.15.96:0/1282592293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff601df8640 1 -- 172.21.15.96:0/1282592293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec009420 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff601df8640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c3d0 tx=0x7ff5f800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5f8005ce0 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f8005e40 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5fc0a5c30 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff5f8007220 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5fc0a5670 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5fc134ab0 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7ff5f8007440 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 0x7ff5e0037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7ff5f8046a60 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff6015f7640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 0x7ff5e0037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.088+0000 7ff6015f7640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 0x7ff5e0037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0ad000 tx=0x7ff5ec008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.092+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff5f8019890 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.236+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5fc0a5c30 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.236+0000 7ff5f27fc640 1 -- 172.21.15.96:0/1282592293 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5fc0a5c30 con 0x7ff5fc06f6d0 2024-09-13T17:43:53.242 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:43:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 msgr2=0x7ff5e0037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 0x7ff5e0037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0ad000 tx=0x7ff5ec008040 comp rx=0 tx=0).stop 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 msgr2=0x7ff5fc11cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff5f800c3d0 tx=0x7ff5f800c890 comp rx=0 tx=0).stop 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 shutdown_connections 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff5e00352a0 0x7ff5e0037760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 --2- 172.21.15.96:0/1282592293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5fc06f6d0 0x7ff5fc11cd70 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 >> 172.21.15.96:0/1282592293 conn(0x7ff5fc098a60 msgr2=0x7ff5fc139420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 shutdown_connections 2024-09-13T17:43:53.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.240+0000 7ff60405c640 1 -- 172.21.15.96:0/1282592293 wait complete. 2024-09-13T17:43:53.258 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:43:53.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.372+0000 7f4b412e8640 1 Processor -- start 2024-09-13T17:43:53.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.372+0000 7f4b412e8640 1 -- start start 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b3c13af50 0x7f4b3c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b3c138050 con 0x7f4b3c09d690 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b3c09c410 con 0x7f4b3c13af50 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b3c13af50 0x7f4b3c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4b3c13af50 0x7f4b3c13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48718/0 (socket says 172.21.15.96:48718) 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 -- 172.21.15.96:0/1749112856 learned_addr learned my addr 172.21.15.96:0/1749112856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:53.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 -- 172.21.15.96:0/1749112856 >> v1:172.21.15.96:6789/0 conn(0x7f4b3c09d690 legacy=0x7f4b3c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 -- 172.21.15.96:0/1749112856 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b3c09cba0 con 0x7f4b3c13af50 2024-09-13T17:43:53.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b33fff640 1 --2- 172.21.15.96:0/1749112856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c13af50 0x7f4b3c13d340 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4b240097f0 tx=0x7f4b2401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cecd8878f496e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b3a7fc640 1 -- 172.21.15.96:0/1749112856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b240a8900 con 0x7f4b3c13af50 2024-09-13T17:43:53.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b3a7fc640 1 -- 172.21.15.96:0/1749112856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b240a8a60 con 0x7f4b3c13af50 2024-09-13T17:43:53.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b3a7fc640 1 -- 172.21.15.96:0/1749112856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b240a8d20 con 0x7f4b3c13af50 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- 172.21.15.96:0/1749112856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c13af50 msgr2=0x7f4b3c13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 --2- 172.21.15.96:0/1749112856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c13af50 0x7f4b3c13d340 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f4b240097f0 tx=0x7f4b2401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- 172.21.15.96:0/1749112856 shutdown_connections 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 --2- 172.21.15.96:0/1749112856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c13af50 0x7f4b3c13d340 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- 172.21.15.96:0/1749112856 >> 172.21.15.96:0/1749112856 conn(0x7f4b3c06bc60 msgr2=0x7f4b3c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- 172.21.15.96:0/1749112856 shutdown_connections 2024-09-13T17:43:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.376+0000 7f4b412e8640 1 -- 172.21.15.96:0/1749112856 wait complete. 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 Processor -- start 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 -- start start 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b3c0a4f90 con 0x7f4b3c09d690 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b3affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48726/0 (socket says 172.21.15.96:48726) 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b3affd640 1 -- 172.21.15.96:0/3869113166 learned_addr learned my addr 172.21.15.96:0/3869113166 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b3affd640 1 -- 172.21.15.96:0/3869113166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240094d0 con 0x7f4b3c09d690 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b3affd640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c340 tx=0x7f4b2c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b2c005150 con 0x7f4b3c09d690 2024-09-13T17:43:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b2c0052b0 con 0x7f4b3c09d690 2024-09-13T17:43:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b3c0a2c20 con 0x7f4b3c09d690 2024-09-13T17:43:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4b2c007030 con 0x7f4b3c09d690 2024-09-13T17:43:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3c1373b0 con 0x7f4b3c09d690 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7f4b2c007190 con 0x7f4b3c09d690 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b3c0a3f70 con 0x7f4b3c09d690 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 0x7f4b1c0377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f4b2c046710 con 0x7f4b3c09d690 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b33fff640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 0x7f4b1c0377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.380+0000 7f4b33fff640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 0x7f4b1c0377b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b240b2000 tx=0x7f4b240aa070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.384+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4b3c0a3f70 con 0x7f4b3c09d690 2024-09-13T17:43:53.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.544+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b3c138050 con 0x7f4b3c09d690 2024-09-13T17:43:53.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.544+0000 7f4b337fe640 1 -- 172.21.15.96:0/3869113166 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4b3c138050 con 0x7f4b3c09d690 2024-09-13T17:43:53.548 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 msgr2=0x7f4b1c0377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 0x7f4b1c0377b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4b240b2000 tx=0x7f4b240aa070 comp rx=0 tx=0).stop 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 msgr2=0x7f4b3c1c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c00c340 tx=0x7f4b2c00c800 comp rx=0 tx=0).stop 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 shutdown_connections 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f4b1c0352f0 0x7f4b1c0377b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 --2- 172.21.15.96:0/3869113166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4b3c09d690 0x7f4b3c1c9820 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 >> 172.21.15.96:0/3869113166 conn(0x7f4b3c06bc60 msgr2=0x7f4b3c131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 shutdown_connections 2024-09-13T17:43:53.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.548+0000 7f4b412e8640 1 -- 172.21.15.96:0/3869113166 wait complete. 2024-09-13T17:43:53.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:43:53.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aabd06640 1 Processor -- start 2024-09-13T17:43:53.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aabd06640 1 -- start start 2024-09-13T17:43:53.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aabd06640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1aa413af50 0x7f1aa413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aabd06640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aa4138050 con 0x7f1aa409d690 2024-09-13T17:43:53.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aabd06640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aa409c410 con 0x7f1aa413af50 2024-09-13T17:43:53.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa92a1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1aa413af50 0x7f1aa413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa92a1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1aa413af50 0x7f1aa413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48768/0 (socket says 172.21.15.96:48768) 2024-09-13T17:43:53.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa9aa2640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1aa409d690 0x7f1aa4138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57616/0 (socket says 172.21.15.96:57616) 2024-09-13T17:43:53.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa92a1640 1 -- 172.21.15.96:0/2014151471 learned_addr learned my addr 172.21.15.96:0/2014151471 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:53.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa92a1640 1 -- 172.21.15.96:0/2014151471 >> v1:172.21.15.96:6789/0 conn(0x7f1aa409d690 legacy=0x7f1aa4138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.692+0000 7f1aa92a1640 1 -- 172.21.15.96:0/2014151471 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa409cba0 con 0x7f1aa413af50 2024-09-13T17:43:53.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa92a1640 1 --2- 172.21.15.96:0/2014151471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa413af50 0x7f1aa413d340 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1a940097f0 tx=0x7f1a9401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5f4e4e76a18f45 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa8aa0640 1 -- 172.21.15.96:0/2014151471 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a940a8900 con 0x7f1aa413af50 2024-09-13T17:43:53.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa8aa0640 1 -- 172.21.15.96:0/2014151471 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a940a8a60 con 0x7f1aa413af50 2024-09-13T17:43:53.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa8aa0640 1 -- 172.21.15.96:0/2014151471 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a940a8d20 con 0x7f1aa413af50 2024-09-13T17:43:53.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/2014151471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa413af50 msgr2=0x7f1aa413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 --2- 172.21.15.96:0/2014151471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa413af50 0x7f1aa413d340 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1a940097f0 tx=0x7f1a9401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/2014151471 shutdown_connections 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 --2- 172.21.15.96:0/2014151471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa413af50 0x7f1aa413d340 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/2014151471 >> 172.21.15.96:0/2014151471 conn(0x7f1aa406bc60 msgr2=0x7f1aa406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/2014151471 shutdown_connections 2024-09-13T17:43:53.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/2014151471 wait complete. 2024-09-13T17:43:53.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 Processor -- start 2024-09-13T17:43:53.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- start start 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa40a4f90 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa9aa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa9aa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48784/0 (socket says 172.21.15.96:48784) 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa9aa2640 1 -- 172.21.15.96:0/4139788318 learned_addr learned my addr 172.21.15.96:0/4139788318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa9aa2640 1 -- 172.21.15.96:0/4139788318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a940094d0 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aa9aa2640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c3b0 tx=0x7f1aa000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa0007410 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa0007570 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa40a2c20 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1aa00047f0 con 0x7f1aa409d690 2024-09-13T17:43:53.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.696+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa41373b0 con 0x7f1aa409d690 2024-09-13T17:43:53.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7f1aa0004950 con 0x7f1aa409d690 2024-09-13T17:43:53.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa40a3f70 con 0x7f1aa409d690 2024-09-13T17:43:53.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1a9a7fc640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 0x7f1a880377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:53.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f1aa0046be0 con 0x7f1aa409d690 2024-09-13T17:43:53.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1aa92a1640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 0x7f1a880377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:53.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.700+0000 7f1aa92a1640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 0x7f1a880377b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a940b2000 tx=0x7f1a940aa070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:53.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.704+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1aa40a3f70 con 0x7f1aa409d690 2024-09-13T17:43:53.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1aa4138050 con 0x7f1aa409d690 2024-09-13T17:43:53.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1a9a7fc640 1 -- 172.21.15.96:0/4139788318 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1aa4138050 con 0x7f1aa409d690 2024-09-13T17:43:53.864 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 msgr2=0x7f1a880377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 0x7f1a880377b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1a940b2000 tx=0x7f1a940aa070 comp rx=0 tx=0).stop 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 msgr2=0x7f1aa41c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1aa000c3b0 tx=0x7f1aa000c870 comp rx=0 tx=0).stop 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 shutdown_connections 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1a880352f0 0x7f1a880377b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 --2- 172.21.15.96:0/4139788318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1aa409d690 0x7f1aa41c9820 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:53.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 >> 172.21.15.96:0/4139788318 conn(0x7f1aa406bc60 msgr2=0x7f1aa4131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:53.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 shutdown_connections 2024-09-13T17:43:53.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.860+0000 7f1aabd06640 1 -- 172.21.15.96:0/4139788318 wait complete. 2024-09-13T17:43:53.882 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:43:54.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.996+0000 7f66a2ca4640 1 Processor -- start 2024-09-13T17:43:54.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:53.996+0000 7f66a2ca4640 1 -- start start 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f669c13af50 0x7f669c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c138050 con 0x7f669c09d690 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c09c410 con 0x7f669c13af50 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f669c13af50 0x7f669c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f669c13af50 0x7f669c13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48798/0 (socket says 172.21.15.96:48798) 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a0a40640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f669c09d690 0x7f669c138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57618/0 (socket says 172.21.15.96:57618) 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 -- 172.21.15.96:0/3748548813 learned_addr learned my addr 172.21.15.96:0/3748548813 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 -- 172.21.15.96:0/3748548813 >> v1:172.21.15.96:6789/0 conn(0x7f669c09d690 legacy=0x7f669c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 -- 172.21.15.96:0/3748548813 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c09cba0 con 0x7f669c13af50 2024-09-13T17:43:54.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669bfff640 1 --2- 172.21.15.96:0/3748548813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c13af50 0x7f669c13d340 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f668c0097f0 tx=0x7f668c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd25b8903a86080 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669b7fe640 1 -- 172.21.15.96:0/3748548813 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f668c0a8900 con 0x7f669c13af50 2024-09-13T17:43:54.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669b7fe640 1 -- 172.21.15.96:0/3748548813 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c0a8a60 con 0x7f669c13af50 2024-09-13T17:43:54.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f669b7fe640 1 -- 172.21.15.96:0/3748548813 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f668c0a8d20 con 0x7f669c13af50 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3748548813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c13af50 msgr2=0x7f669c13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3748548813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c13af50 0x7f669c13d340 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f668c0097f0 tx=0x7f668c01ecd0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3748548813 shutdown_connections 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3748548813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c13af50 0x7f669c13d340 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3748548813 >> 172.21.15.96:0/3748548813 conn(0x7f669c06bc60 msgr2=0x7f669c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3748548813 shutdown_connections 2024-09-13T17:43:54.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.000+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3748548813 wait complete. 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 Processor -- start 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 -- start start 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0a2d40 con 0x7f669c09d690 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a0a40640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a0a40640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48804/0 (socket says 172.21.15.96:48804) 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a0a40640 1 -- 172.21.15.96:0/3266299456 learned_addr learned my addr 172.21.15.96:0/3266299456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:54.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a0a40640 1 -- 172.21.15.96:0/3266299456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0094d0 con 0x7f669c09d690 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a0a40640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f668400c3b0 tx=0x7f668400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6684007410 con 0x7f669c09d690 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6684007570 con 0x7f669c09d690 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c1372e0 con 0x7f669c09d690 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66840047f0 con 0x7f669c09d690 2024-09-13T17:43:54.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c0a5100 con 0x7f669c09d690 2024-09-13T17:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7f6684004950 con 0x7f669c09d690 2024-09-13T17:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c09da70 con 0x7f669c09d690 2024-09-13T17:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 0x7f667c0377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f6684046da0 con 0x7f669c09d690 2024-09-13T17:43:54.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f669bfff640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 0x7f667c0377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.004+0000 7f669bfff640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 0x7f667c0377b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b2000 tx=0x7f668c0aa070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.008+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66840192c0 con 0x7f669c09d690 2024-09-13T17:43:54.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.168+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f669c138050 con 0x7f669c09d690 2024-09-13T17:43:54.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66997fa640 1 -- 172.21.15.96:0/3266299456 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+207 (secure 0 0 0) 0x7f669c138050 con 0x7f669c09d690 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 6, 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": "cephfs_metadata" 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:43:54.175 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 7, 2024-09-13T17:43:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": "cephfs_data" 2024-09-13T17:43:54.176 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:43:54.176 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:43:54.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 msgr2=0x7f667c0377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 0x7f667c0377b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f668c0b2000 tx=0x7f668c0aa070 comp rx=0 tx=0).stop 2024-09-13T17:43:54.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 msgr2=0x7f669c1c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f668400c3b0 tx=0x7f668400c870 comp rx=0 tx=0).stop 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 shutdown_connections 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f667c0352f0 0x7f667c0377b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 --2- 172.21.15.96:0/3266299456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f669c09d690 0x7f669c1c9550 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 >> 172.21.15.96:0/3266299456 conn(0x7f669c06bc60 msgr2=0x7f669c131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 shutdown_connections 2024-09-13T17:43:54.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.172+0000 7f66a2ca4640 1 -- 172.21.15.96:0/3266299456 wait complete. 2024-09-13T17:43:54.193 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs_b' 2024-09-13T17:43:54.193 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_b_metadata --pg_num_min 64 2024-09-13T17:43:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad27e29640 1 Processor -- start 2024-09-13T17:43:54.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad27e29640 1 -- start start 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad27e29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fad2009e890 0x7fad2013ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad27e29640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2009e0d0 con 0x7fad20135e80 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad27e29640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2009ef50 con 0x7fad2009e890 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad253c4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fad2009e890 0x7fad2013ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad25bc5640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fad20135e80 0x7fad20136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57622/0 (socket says 172.21.15.96:57622) 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad25bc5640 1 -- 172.21.15.96:0/2747041836 learned_addr learned my addr 172.21.15.96:0/2747041836 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:54.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad253c4640 1 -- 172.21.15.96:0/2747041836 >> v1:172.21.15.96:6789/0 conn(0x7fad20135e80 legacy=0x7fad20136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad253c4640 1 -- 172.21.15.96:0/2747041836 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2009bf40 con 0x7fad2009e890 2024-09-13T17:43:54.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad24bc3640 1 -- 172.21.15.96:0/2747041836 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010422036 0 0) 0x7fad2009e0d0 con 0x7fad20135e80 2024-09-13T17:43:54.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.312+0000 7fad253c4640 1 --2- 172.21.15.96:0/2747041836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad2013ade0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009740 tx=0x7fad1c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728364dc5699baf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad24bc3640 1 -- 172.21.15.96:0/2747041836 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad1c0ab570 con 0x7fad2009e890 2024-09-13T17:43:54.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad24bc3640 1 -- 172.21.15.96:0/2747041836 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c0a7070 con 0x7fad2009e890 2024-09-13T17:43:54.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad24bc3640 1 -- 172.21.15.96:0/2747041836 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad1c0ab820 con 0x7fad2009e890 2024-09-13T17:43:54.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/2747041836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 msgr2=0x7fad2013ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 --2- 172.21.15.96:0/2747041836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad2013ade0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad1c009740 tx=0x7fad1c01c420 comp rx=0 tx=0).stop 2024-09-13T17:43:54.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/2747041836 shutdown_connections 2024-09-13T17:43:54.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 --2- 172.21.15.96:0/2747041836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad2013ade0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/2747041836 >> 172.21.15.96:0/2747041836 conn(0x7fad20098a60 msgr2=0x7fad2009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/2747041836 shutdown_connections 2024-09-13T17:43:54.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/2747041836 wait complete. 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 Processor -- start 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- start start 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200a3c80 con 0x7fad2009e890 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad25bc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad25bc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48830/0 (socket says 172.21.15.96:48830) 2024-09-13T17:43:54.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad25bc5640 1 -- 172.21.15.96:0/134872677 learned_addr learned my addr 172.21.15.96:0/134872677 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad25bc5640 1 -- 172.21.15.96:0/134872677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009420 con 0x7fad2009e890 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad25bc5640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c390 tx=0x7fad1400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad14007410 con 0x7fad2009e890 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14007590 con 0x7fad2009e890 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fad140047f0 con 0x7fad2009e890 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200a21c0 con 0x7fad2009e890 2024-09-13T17:43:54.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.316+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad201c9b40 con 0x7fad2009e890 2024-09-13T17:43:54.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) v1 ==== 43154+0+0 (secure 0 0 0) 0x7fad14004950 con 0x7fad2009e890 2024-09-13T17:43:54.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad200a6380 con 0x7fad2009e890 2024-09-13T17:43:54.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad127fc640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 0x7fad040377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 3783+0+0 (secure 0 0 0) 0x7fad140472e0 con 0x7fad2009e890 2024-09-13T17:43:54.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad253c4640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 0x7fad040377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.320+0000 7fad253c4640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 0x7fad040377b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0af000 tx=0x7fad1c008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.324+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fad200a6380 con 0x7fad2009e890 2024-09-13T17:43:54.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.480+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_b_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fad200a6380 con 0x7fad2009e890 2024-09-13T17:43:54.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.612+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 19) v1 ==== 43362+0+0 (secure 0 0 0) 0x7fad140139d0 con 0x7fad2009e890 2024-09-13T17:43:54.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.616+0000 7fad127fc640 1 -- 172.21.15.96:0/134872677 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_b_metadata", "pg_num_min": 64}]=0 pool 'cephfs_b_metadata' created v39) v1 ==== 142+0+0 (secure 0 0 0) 0x7fad1401c800 con 0x7fad2009e890 2024-09-13T17:43:54.622 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_b_metadata' created 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 msgr2=0x7fad040377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 0x7fad040377b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0af000 tx=0x7fad1c008040 comp rx=0 tx=0).stop 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 msgr2=0x7fad200a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c390 tx=0x7fad1400c850 comp rx=0 tx=0).stop 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 shutdown_connections 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fad040352f0 0x7fad040377b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 --2- 172.21.15.96:0/134872677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fad2009e890 0x7fad200a2ec0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.620+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 >> 172.21.15.96:0/134872677 conn(0x7fad20098a60 msgr2=0x7fad2009a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.624+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 shutdown_connections 2024-09-13T17:43:54.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.624+0000 7fad27e29640 1 -- 172.21.15.96:0/134872677 wait complete. 2024-09-13T17:43:54.653 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_b_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:43:54.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd18a72e640 1 Processor -- start 2024-09-13T17:43:54.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd18a72e640 1 -- start start 2024-09-13T17:43:54.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd18a72e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd18409e820 0x7fd18413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd18a72e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1840a68b0 con 0x7fd184134e70 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd18a72e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18409ddc0 con 0x7fd18409e820 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd1837fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd18409e820 0x7fd18413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd1837fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd18409e820 0x7fd18413ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48840/0 (socket says 172.21.15.96:48840) 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd1837fe640 1 -- 172.21.15.96:0/1630072596 learned_addr learned my addr 172.21.15.96:0/1630072596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd1837fe640 1 -- 172.21.15.96:0/1630072596 >> v1:172.21.15.96:6789/0 conn(0x7fd184134e70 legacy=0x7fd184135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd182ffd640 1 -- 172.21.15.96:0/1630072596 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080698685 0 0) 0x7fd1840a68b0 con 0x7fd184134e70 2024-09-13T17:43:54.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.772+0000 7fd1837fe640 1 -- 172.21.15.96:0/1630072596 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184136970 con 0x7fd18409e820 2024-09-13T17:43:54.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd1837fe640 1 --2- 172.21.15.96:0/1630072596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd18413ae50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097f0 tx=0x7fd17801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fe3381f9d72549 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd182ffd640 1 -- 172.21.15.96:0/1630072596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1780ac540 con 0x7fd18409e820 2024-09-13T17:43:54.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd182ffd640 1 -- 172.21.15.96:0/1630072596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780ac6a0 con 0x7fd18409e820 2024-09-13T17:43:54.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd182ffd640 1 -- 172.21.15.96:0/1630072596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1780ac960 con 0x7fd18409e820 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- 172.21.15.96:0/1630072596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 msgr2=0x7fd18413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1630072596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd18413ae50 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd1780097f0 tx=0x7fd17801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- 172.21.15.96:0/1630072596 shutdown_connections 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1630072596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd18413ae50 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- 172.21.15.96:0/1630072596 >> 172.21.15.96:0/1630072596 conn(0x7fd184098a60 msgr2=0x7fd18409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:54.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- 172.21.15.96:0/1630072596 shutdown_connections 2024-09-13T17:43:54.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- 172.21.15.96:0/1630072596 wait complete. 2024-09-13T17:43:54.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 Processor -- start 2024-09-13T17:43:54.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- start start 2024-09-13T17:43:54.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd18a72e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840a3320 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd183fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd183fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48846/0 (socket says 172.21.15.96:48846) 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.776+0000 7fd183fff640 1 -- 172.21.15.96:0/1696975275 learned_addr learned my addr 172.21.15.96:0/1696975275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd183fff640 1 -- 172.21.15.96:0/1696975275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780094d0 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd183fff640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd17c001570 tx=0x7fd17c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd17c003bb0 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c003d30 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd184132820 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd17c0051e0 con 0x7fd18409e820 2024-09-13T17:43:54.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1840a34f0 con 0x7fd18409e820 2024-09-13T17:43:54.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 43362+0+0 (secure 0 0 0) 0x7fd17c005340 con 0x7fd18409e820 2024-09-13T17:43:54.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd18413a390 con 0x7fd18409e820 2024-09-13T17:43:54.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 0x7fd1700379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:54.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4155+0+0 (secure 0 0 0) 0x7fd17c047300 con 0x7fd18409e820 2024-09-13T17:43:54.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd1837fe640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 0x7fd1700379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:54.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.780+0000 7fd1837fe640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 0x7fd1700379f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ae000 tx=0x7fd178008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:54.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.784+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd17c0196d0 con 0x7fd18409e820 2024-09-13T17:43:54.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:54.932+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd1840a3dd0 con 0x7fd18409e820 2024-09-13T17:43:55.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.632+0000 7fd180ff9640 1 -- 172.21.15.96:0/1696975275 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_b_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_b_data' created v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd17c010300 con 0x7fd18409e820 2024-09-13T17:43:55.636 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_b_data' created 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 msgr2=0x7fd1700379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 0x7fd1700379f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1780ae000 tx=0x7fd178008040 comp rx=0 tx=0).stop 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 msgr2=0x7fd1840a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd17c001570 tx=0x7fd17c00cc80 comp rx=0 tx=0).stop 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 shutdown_connections 2024-09-13T17:43:55.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd170035530 0x7fd1700379f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 --2- 172.21.15.96:0/1696975275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd18409e820 0x7fd1840a40a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.636+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 >> 172.21.15.96:0/1696975275 conn(0x7fd184098a60 msgr2=0x7fd184098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.640+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 shutdown_connections 2024-09-13T17:43:55.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.640+0000 7fd18a72e640 1 -- 172.21.15.96:0/1696975275 wait complete. 2024-09-13T17:43:55.682 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs_b cephfs_b_metadata cephfs_b_data 2024-09-13T17:43:55.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ceb9c6640 1 Processor -- start 2024-09-13T17:43:55.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ceb9c6640 1 -- start start 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ceb9c6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ce4134e70 0x7f6ce409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ceb9c6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce409ed60 con 0x7f6ce4135250 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ceb9c6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce409ddc0 con 0x7f6ce4134e70 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ce9762640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6ce4134e70 0x7f6ce409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ce8f61640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6ce4135250 0x7f6ce4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57646/0 (socket says 172.21.15.96:57646) 2024-09-13T17:43:55.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.796+0000 7f6ce8f61640 1 -- 172.21.15.96:0/2102276584 learned_addr learned my addr 172.21.15.96:0/2102276584 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:55.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ce9762640 1 -- 172.21.15.96:0/2102276584 >> v1:172.21.15.96:6789/0 conn(0x7f6ce4135250 legacy=0x7f6ce4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ce9762640 1 -- 172.21.15.96:0/2102276584 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce40a68b0 con 0x7f6ce4134e70 2024-09-13T17:43:55.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ce9762640 1 --2- 172.21.15.96:0/2102276584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce409e820 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873f11885e4edf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6cdbfff640 1 -- 172.21.15.96:0/2102276584 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0a57b0 con 0x7f6ce4134e70 2024-09-13T17:43:55.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6cdbfff640 1 -- 172.21.15.96:0/2102276584 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc0a5910 con 0x7f6ce4134e70 2024-09-13T17:43:55.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6cdbfff640 1 -- 172.21.15.96:0/2102276584 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0a5bd0 con 0x7f6ce4134e70 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2102276584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 msgr2=0x7f6ce409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2102276584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce409e820 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009d90 tx=0x7f6cdc01c970 comp rx=0 tx=0).stop 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2102276584 shutdown_connections 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2102276584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce409e820 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2102276584 >> 172.21.15.96:0/2102276584 conn(0x7f6ce4098a60 msgr2=0x7f6ce409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2102276584 shutdown_connections 2024-09-13T17:43:55.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.800+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2102276584 wait complete. 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 Processor -- start 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 -- start start 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce40a0420 con 0x7f6ce4134e70 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce9762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce9762640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48850/0 (socket says 172.21.15.96:48850) 2024-09-13T17:43:55.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce9762640 1 -- 172.21.15.96:0/2330413161 learned_addr learned my addr 172.21.15.96:0/2330413161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:55.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce9762640 1 -- 172.21.15.96:0/2330413161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc009a70 con 0x7f6ce4134e70 2024-09-13T17:43:55.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce9762640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0b0040 tx=0x7f6cdc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0b9790 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc0b98f0 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce40a2210 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6cdc0b9bd0 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce40a0f50 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce41368b0 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 43362+0+0 (secure 0 0 0) 0x7f6cdc0b9d30 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 0x7f6cc80379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4539+0+0 (secure 0 0 0) 0x7f6cdc0e3280 con 0x7f6ce4134e70 2024-09-13T17:43:55.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce8f61640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 0x7f6cc80379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:55.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.804+0000 7f6ce8f61640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 0x7f6cc80379f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4000f70 tx=0x7f6cd4007a90 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:55.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:55.812+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6cdc0abd10 con 0x7f6ce4134e70 2024-09-13T17:43:56.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.000+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs_b", "metadata": "cephfs_b_metadata", "data": "cephfs_b_data"} v 0) v1 -- 0x7f6ce40a61e0 con 0x7f6ce4134e70 2024-09-13T17:43:56.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.008+0000 7f6cd9ffb640 1 -- 172.21.15.96:0/2330413161 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs_b", "metadata": "cephfs_b_metadata", "data": "cephfs_b_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6cdc0b20c0 con 0x7f6ce4134e70 2024-09-13T17:43:56.017 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 8 and data pool 9 2024-09-13T17:43:56.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.020+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 msgr2=0x7f6cc80379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.020+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 0x7f6cc80379f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6cd4000f70 tx=0x7f6cd4007a90 comp rx=0 tx=0).stop 2024-09-13T17:43:56.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.020+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 msgr2=0x7f6ce413b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.020+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0b0040 tx=0x7f6cdc019710 comp rx=0 tx=0).stop 2024-09-13T17:43:56.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 shutdown_connections 2024-09-13T17:43:56.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6cc8035530 0x7f6cc80379f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 --2- 172.21.15.96:0/2330413161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ce4134e70 0x7f6ce413b360 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 >> 172.21.15.96:0/2330413161 conn(0x7f6ce4098a60 msgr2=0x7f6ce4099110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 shutdown_connections 2024-09-13T17:43:56.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.024+0000 7f6ceb9c6640 1 -- 172.21.15.96:0/2330413161 wait complete. 2024-09-13T17:43:56.066 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:56.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.172+0000 7f5907f54640 1 Processor -- start 2024-09-13T17:43:56.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.172+0000 7f5907f54640 1 -- start start 2024-09-13T17:43:56.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5907f54640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5900134e70 0x7f590009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5907f54640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590009ed60 con 0x7f5900135250 2024-09-13T17:43:56.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5907f54640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f590009ddc0 con 0x7f5900134e70 2024-09-13T17:43:56.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5905cf0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5900134e70 0x7f590009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f59054ef640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5900135250 0x7f5900135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57650/0 (socket says 172.21.15.96:57650) 2024-09-13T17:43:56.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f59054ef640 1 -- 172.21.15.96:0/1465184945 learned_addr learned my addr 172.21.15.96:0/1465184945 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:56.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5905cf0640 1 -- 172.21.15.96:0/1465184945 >> v1:172.21.15.96:6789/0 conn(0x7f5900135250 legacy=0x7f5900135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5905cf0640 1 -- 172.21.15.96:0/1465184945 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000a68b0 con 0x7f5900134e70 2024-09-13T17:43:56.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5904cee640 1 -- 172.21.15.96:0/1465184945 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669198309 0 0) 0x7f58f80014a0 con 0x7f5900135250 2024-09-13T17:43:56.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5905cf0640 1 --2- 172.21.15.96:0/1465184945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590009e820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009740 tx=0x7f58f401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40b7777b972e052e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5904cee640 1 -- 172.21.15.96:0/1465184945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f40ab530 con 0x7f5900134e70 2024-09-13T17:43:56.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5904cee640 1 -- 172.21.15.96:0/1465184945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f40a7050 con 0x7f5900134e70 2024-09-13T17:43:56.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.176+0000 7f5904cee640 1 -- 172.21.15.96:0/1465184945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f40ab7e0 con 0x7f5900134e70 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/1465184945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 msgr2=0x7f590009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 --2- 172.21.15.96:0/1465184945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590009e820 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009740 tx=0x7f58f401c420 comp rx=0 tx=0).stop 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/1465184945 shutdown_connections 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 --2- 172.21.15.96:0/1465184945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590009e820 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/1465184945 >> 172.21.15.96:0/1465184945 conn(0x7f5900098a60 msgr2=0x7f590009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/1465184945 shutdown_connections 2024-09-13T17:43:56.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/1465184945 wait complete. 2024-09-13T17:43:56.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 Processor -- start 2024-09-13T17:43:56.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- start start 2024-09-13T17:43:56.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900132f50 con 0x7f5900134e70 2024-09-13T17:43:56.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5905cf0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5905cf0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48878/0 (socket says 172.21.15.96:48878) 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5905cf0640 1 -- 172.21.15.96:0/328135048 learned_addr learned my addr 172.21.15.96:0/328135048 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5905cf0640 1 -- 172.21.15.96:0/328135048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f4009420 con 0x7f5900134e70 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5905cf0640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f58f4019030 tx=0x7f58f401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f40b7450 con 0x7f5900134e70 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f40b75b0 con 0x7f5900134e70 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59000a1ff0 con 0x7f5900134e70 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58f40b7890 con 0x7f5900134e70 2024-09-13T17:43:56.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590013ba70 con 0x7f5900134e70 2024-09-13T17:43:56.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.180+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59001368b0 con 0x7f5900134e70 2024-09-13T17:43:56.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.184+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 43362+0+0 (secure 0 0 0) 0x7f58f40b79f0 con 0x7f5900134e70 2024-09-13T17:43:56.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.184+0000 7f58f27fc640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 0x7f58e40379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.184+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f58f40e2c70 con 0x7f5900134e70 2024-09-13T17:43:56.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.184+0000 7f59054ef640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 0x7f58e40379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.184+0000 7f59054ef640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 0x7f58e40379f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000f90 tx=0x7f58f80016a0 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.188+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58f40bf590 con 0x7f5900134e70 2024-09-13T17:43:56.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.348+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59000a2740 con 0x7f5900134e70 2024-09-13T17:43:56.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.348+0000 7f58f27fc640 1 -- 172.21.15.96:0/328135048 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+14778 (secure 0 0 0) 0x7f58f40b0020 con 0x7f5900134e70 2024-09-13T17:43:56.353 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:56.356 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":41,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:56.007109+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 msgr2=0x7f58e40379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 0x7f58e40379f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f58f8000f90 tx=0x7f58f80016a0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 msgr2=0x7f590013aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f58f4019030 tx=0x7f58f401ac60 comp rx=0 tx=0).stop 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 shutdown_connections 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f58e4035530 0x7f58e40379f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 --2- 172.21.15.96:0/328135048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5900134e70 0x7f590013aff0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 >> 172.21.15.96:0/328135048 conn(0x7f5900098a60 msgr2=0x7f5900098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 shutdown_connections 2024-09-13T17:43:56.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.352+0000 7f5907f54640 1 -- 172.21.15.96:0/328135048 wait complete. 2024-09-13T17:43:56.372 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:56.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:43:56.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff904ba4640 1 Processor -- start 2024-09-13T17:43:56.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff904ba4640 1 -- start start 2024-09-13T17:43:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff904ba4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff90006f6c0 0x7ff9000a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff904ba4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90006eee0 con 0x7ff90006faa0 2024-09-13T17:43:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff904ba4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff900070090 con 0x7ff90006f6c0 2024-09-13T17:43:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff8fe59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff90006f6c0 0x7ff9000a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff8f5d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff90006faa0 0x7ff9000a19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57662/0 (socket says 172.21.15.96:57662) 2024-09-13T17:43:56.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff8f5d9b640 1 -- 172.21.15.96:0/1455695694 learned_addr learned my addr 172.21.15.96:0/1455695694 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:56.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff8fe59c640 1 -- 172.21.15.96:0/1455695694 >> v1:172.21.15.96:6789/0 conn(0x7ff90006faa0 legacy=0x7ff9000a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.488+0000 7ff8fe59c640 1 -- 172.21.15.96:0/1455695694 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000a3040 con 0x7ff90006f6c0 2024-09-13T17:43:56.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff8fe59c640 1 --2- 172.21.15.96:0/1455695694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff9000a1470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009770 tx=0x7ff8f001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72eb112381b9dce2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff8fdd9b640 1 -- 172.21.15.96:0/1455695694 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f00ab530 con 0x7ff90006f6c0 2024-09-13T17:43:56.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff8fdd9b640 1 -- 172.21.15.96:0/1455695694 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f00a7050 con 0x7ff90006f6c0 2024-09-13T17:43:56.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff8fdd9b640 1 -- 172.21.15.96:0/1455695694 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f00ab7e0 con 0x7ff90006f6c0 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- 172.21.15.96:0/1455695694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 msgr2=0x7ff9000a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 --2- 172.21.15.96:0/1455695694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff9000a1470 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009770 tx=0x7ff8f001c390 comp rx=0 tx=0).stop 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- 172.21.15.96:0/1455695694 shutdown_connections 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 --2- 172.21.15.96:0/1455695694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff9000a1470 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- 172.21.15.96:0/1455695694 >> 172.21.15.96:0/1455695694 conn(0x7ff90006a150 msgr2=0x7ff90006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- 172.21.15.96:0/1455695694 shutdown_connections 2024-09-13T17:43:56.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- 172.21.15.96:0/1455695694 wait complete. 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 Processor -- start 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.492+0000 7ff904ba4640 1 -- start start 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff904ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff904ba4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000a0430 con 0x7ff90006f6c0 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8fe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8fe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48888/0 (socket says 172.21.15.96:48888) 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8fe59c640 1 -- 172.21.15.96:0/2595377900 learned_addr learned my addr 172.21.15.96:0/2595377900 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:56.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8fe59c640 1 -- 172.21.15.96:0/2595377900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0009420 con 0x7ff90006f6c0 2024-09-13T17:43:56.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8fe59c640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0019fc0 tx=0x7ff8f001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f00b7450 con 0x7ff90006f6c0 2024-09-13T17:43:56.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f00b75b0 con 0x7ff90006f6c0 2024-09-13T17:43:56.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8f00b7870 con 0x7ff90006f6c0 2024-09-13T17:43:56.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90009fbb0 con 0x7ff90006f6c0 2024-09-13T17:43:56.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9000a5a30 con 0x7ff90006f6c0 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) v1 ==== 43362+0+0 (secure 0 0 0) 0x7ff8f00b79d0 con 0x7ff90006f6c0 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9000a1470 con 0x7ff90006f6c0 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 0x7ff8e00379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4643+0+0 (secure 0 0 0) 0x7ff8f00e26a0 con 0x7ff90006f6c0 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f5d9b640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 0x7ff8e00379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.496+0000 7ff8f5d9b640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 0x7ff8e00379f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005890 tx=0x7ff8e8007a90 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.500+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8f00c76c0 con 0x7ff90006f6c0 2024-09-13T17:43:56.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.636+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7ff8f00b5450 con 0x7ff90006f6c0 2024-09-13T17:43:56.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.664+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff90011e1e0 con 0x7ff90006f6c0 2024-09-13T17:43:56.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.664+0000 7ff8f77fe640 1 -- 172.21.15.96:0/2595377900 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+14778 (secure 0 0 0) 0x7ff8f00bb020 con 0x7ff90006f6c0 2024-09-13T17:43:56.670 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:56.670 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":41,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:56.007109+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 msgr2=0x7ff8e00379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 0x7ff8e00379f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8005890 tx=0x7ff8e8007a90 comp rx=0 tx=0).stop 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 msgr2=0x7ff90009df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0019fc0 tx=0x7ff8f001abd0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 shutdown_connections 2024-09-13T17:43:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7ff8e0035530 0x7ff8e00379f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 --2- 172.21.15.96:0/2595377900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff90006f6c0 0x7ff90009df10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 >> 172.21.15.96:0/2595377900 conn(0x7ff90006a150 msgr2=0x7ff9000a62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 shutdown_connections 2024-09-13T17:43:56.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.668+0000 7ff904ba4640 1 -- 172.21.15.96:0/2595377900 wait complete. 2024-09-13T17:43:56.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs_b standby_count_wanted 0 2024-09-13T17:43:56.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d85c6640 1 Processor -- start 2024-09-13T17:43:56.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d85c6640 1 -- start start 2024-09-13T17:43:56.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d85c6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f67d01381b0 0x7f67d0138590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d85c6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d009e040 con 0x7f67d0138ad0 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d85c6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d0099ea0 con 0x7f67d01381b0 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d6362640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f67d01381b0 0x7f67d0138590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d5b61640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f67d0138ad0 0x7f67d006b090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57678/0 (socket says 172.21.15.96:57678) 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d6362640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f67d01381b0 0x7f67d0138590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48898/0 (socket says 172.21.15.96:48898) 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d5b61640 1 -- 172.21.15.96:0/3940353445 learned_addr learned my addr 172.21.15.96:0/3940353445 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d6362640 1 -- 172.21.15.96:0/3940353445 >> v1:172.21.15.96:6789/0 conn(0x7f67d0138ad0 legacy=0x7f67d006b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.848+0000 7f67d6362640 1 -- 172.21.15.96:0/3940353445 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0098be0 con 0x7f67d01381b0 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d6362640 1 --2- 172.21.15.96:0/3940353445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d0138590 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009d90 tx=0x7f67c801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a6d64b15e26af0a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d5360640 1 -- 172.21.15.96:0/3940353445 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80a5740 con 0x7f67d01381b0 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d5360640 1 -- 172.21.15.96:0/3940353445 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80a58a0 con 0x7f67d01381b0 2024-09-13T17:43:56.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d5360640 1 -- 172.21.15.96:0/3940353445 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80a5b60 con 0x7f67d01381b0 2024-09-13T17:43:56.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- 172.21.15.96:0/3940353445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 msgr2=0x7f67d0138590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:56.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 --2- 172.21.15.96:0/3940353445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d0138590 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009d90 tx=0x7f67c801c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- 172.21.15.96:0/3940353445 shutdown_connections 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 --2- 172.21.15.96:0/3940353445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d0138590 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- 172.21.15.96:0/3940353445 >> 172.21.15.96:0/3940353445 conn(0x7f67d009c210 msgr2=0x7f67d0071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- 172.21.15.96:0/3940353445 shutdown_connections 2024-09-13T17:43:56.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- 172.21.15.96:0/3940353445 wait complete. 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 Processor -- start 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- start start 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d85c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d013d9a0 con 0x7f67d01381b0 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d6362640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d6362640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48904/0 (socket says 172.21.15.96:48904) 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.852+0000 7f67d6362640 1 -- 172.21.15.96:0/701564790 learned_addr learned my addr 172.21.15.96:0/701564790 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:56.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d6362640 1 -- 172.21.15.96:0/701564790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8009a70 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d6362640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f67c80194f0 tx=0x7f67c8019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80b9850 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80b99b0 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d013d3e0 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80b9c70 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d0135d60 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f67c80ad440 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d0138590 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 0x7f67bc037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f67c80e3140 con 0x7f67d01381b0 2024-09-13T17:43:56.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d5b61640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 0x7f67bc037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:56.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.856+0000 7f67d5b61640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 0x7f67bc037a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67c4001610 tx=0x7f67c4007ad0 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:56.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:56.860+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f67c80ac3a0 con 0x7f67d01381b0 2024-09-13T17:43:57.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:57.056+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs_b", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f67d0073a50 con 0x7f67d01381b0 2024-09-13T17:43:57.612 INFO:tasks.ceph.mds.a.smithi096.stderr:2024-09-13T17:43:57.608+0000 7f53071ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:43:58.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67ceffd640 1 -- 172.21.15.96:0/701564790 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs_b", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 120+0+0 (secure 0 0 0) 0x7f67c80abb30 con 0x7f67d01381b0 2024-09-13T17:43:58.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 msgr2=0x7f67bc037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 0x7f67bc037a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f67c4001610 tx=0x7f67c4007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 msgr2=0x7f67d00a7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f67c80194f0 tx=0x7f67c8019680 comp rx=0 tx=0).stop 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 shutdown_connections 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f67bc0355c0 0x7f67bc037a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 --2- 172.21.15.96:0/701564790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d01381b0 0x7f67d00a7750 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 >> 172.21.15.96:0/701564790 conn(0x7f67d009c210 msgr2=0x7f67d0137280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 shutdown_connections 2024-09-13T17:43:58.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.016+0000 7f67d85c6640 1 -- 172.21.15.96:0/701564790 wait complete. 2024-09-13T17:43:58.041 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:43:58.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs_b allow_standby_replay false 2024-09-13T17:43:58.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.160+0000 7f804deb0640 1 Processor -- start 2024-09-13T17:43:58.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.160+0000 7f804deb0640 1 -- start start 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f804deb0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f804809dc20 0x7f80481351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f804deb0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804806e130 con 0x7f804806f6d0 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f804deb0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80480a6750 con 0x7f804809dc20 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f8046ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f804809dc20 0x7f80481351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80477fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f804806f6d0 0x7f8048134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57684/0 (socket says 172.21.15.96:57684) 2024-09-13T17:43:58.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80477fe640 1 -- 172.21.15.96:0/1656452513 learned_addr learned my addr 172.21.15.96:0/1656452513 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:58.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80467fc640 1 -- 172.21.15.96:0/1656452513 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265368163 0 0) 0x7f804806e130 con 0x7f804806f6d0 2024-09-13T17:43:58.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80467fc640 1 -- 172.21.15.96:0/1656452513 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8030003530 con 0x7f804806f6d0 2024-09-13T17:43:58.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f8046ffd640 1 -- 172.21.15.96:0/1656452513 >> v1:172.21.15.96:6789/0 conn(0x7f804806f6d0 legacy=0x7f8048134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f8046ffd640 1 -- 172.21.15.96:0/1656452513 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80480712e0 con 0x7f804809dc20 2024-09-13T17:43:58.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f8046ffd640 1 --2- 172.21.15.96:0/1656452513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804809dc20 0x7f80481351c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8038009740 tx=0x7f803801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e179d5a68597e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:58.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80467fc640 1 -- 172.21.15.96:0/1656452513 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80380ab400 con 0x7f804809dc20 2024-09-13T17:43:58.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80467fc640 1 -- 172.21.15.96:0/1656452513 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380a7050 con 0x7f804809dc20 2024-09-13T17:43:58.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.164+0000 7f80467fc640 1 -- 172.21.15.96:0/1656452513 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80380ab6b0 con 0x7f804809dc20 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/1656452513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804809dc20 msgr2=0x7f80481351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 --2- 172.21.15.96:0/1656452513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804809dc20 0x7f80481351c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8038009740 tx=0x7f803801c390 comp rx=0 tx=0).stop 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/1656452513 shutdown_connections 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 --2- 172.21.15.96:0/1656452513 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804809dc20 0x7f80481351c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/1656452513 >> 172.21.15.96:0/1656452513 conn(0x7f8048098a60 msgr2=0x7f804809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/1656452513 shutdown_connections 2024-09-13T17:43:58.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/1656452513 wait complete. 2024-09-13T17:43:58.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 Processor -- start 2024-09-13T17:43:58.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- start start 2024-09-13T17:43:58.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80480a1430 con 0x7f804806f6d0 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f80477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f80477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48934/0 (socket says 172.21.15.96:48934) 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f80477fe640 1 -- 172.21.15.96:0/2689735154 learned_addr learned my addr 172.21.15.96:0/2689735154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f80477fe640 1 -- 172.21.15.96:0/2689735154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009420 con 0x7f804806f6d0 2024-09-13T17:43:58.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f80477fe640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c450 tx=0x7f803c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:58.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f803c005c20 con 0x7f804806f6d0 2024-09-13T17:43:58.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c005d80 con 0x7f804806f6d0 2024-09-13T17:43:58.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80480a0dc0 con 0x7f804806f6d0 2024-09-13T17:43:58.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f803c0071a0 con 0x7f804806f6d0 2024-09-13T17:43:58.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80480a1e70 con 0x7f804806f6d0 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f803c007300 con 0x7f804806f6d0 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 0x7f8034037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80480a0130 con 0x7f804806f6d0 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f803c047220 con 0x7f804806f6d0 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.168+0000 7f8046ffd640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 0x7f8034037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:58.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.172+0000 7f8046ffd640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 0x7f8034037a80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80380ad000 tx=0x7f8038008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:58.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.176+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f803c017020 con 0x7f804806f6d0 2024-09-13T17:43:58.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:58.368+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs_b", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8048134ab0 con 0x7f804806f6d0 2024-09-13T17:43:59.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.020+0000 7f802ffff640 1 -- 172.21.15.96:0/2689735154 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs_b", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 124+0+0 (secure 0 0 0) 0x7f803c01b3b0 con 0x7f804806f6d0 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 msgr2=0x7f8034037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 0x7f8034037a80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f80380ad000 tx=0x7f8038008040 comp rx=0 tx=0).stop 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 msgr2=0x7f804812ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c450 tx=0x7f803c00c910 comp rx=0 tx=0).stop 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 shutdown_connections 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f80340355c0 0x7f8034037a80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 --2- 172.21.15.96:0/2689735154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f804806f6d0 0x7f804812ca30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 >> 172.21.15.96:0/2689735154 conn(0x7f8048098a60 msgr2=0x7f804806fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 shutdown_connections 2024-09-13T17:43:59.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.024+0000 7f804deb0640 1 -- 172.21.15.96:0/2689735154 wait complete. 2024-09-13T17:43:59.048 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs_b' 2024-09-13T17:43:59.049 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:43:59.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660f073640 1 Processor -- start 2024-09-13T17:43:59.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660f073640 1 -- start start 2024-09-13T17:43:59.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660f073640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f660806f6d0 0x7f6608134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660f073640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660806e130 con 0x7f660809dc20 2024-09-13T17:43:59.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660f073640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66080a6750 con 0x7f660806f6d0 2024-09-13T17:43:59.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660ce0f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f660806f6d0 0x7f6608134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660ce0f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f660806f6d0 0x7f6608134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48938/0 (socket says 172.21.15.96:48938) 2024-09-13T17:43:59.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660ce0f640 1 -- 172.21.15.96:0/2132348233 learned_addr learned my addr 172.21.15.96:0/2132348233 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:59.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f660ce0f640 1 -- 172.21.15.96:0/2132348233 >> v1:172.21.15.96:6789/0 conn(0x7f660809dc20 legacy=0x7f6608134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.164+0000 7f66077fe640 1 -- 172.21.15.96:0/2132348233 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261441310 0 0) 0x7f660806e130 con 0x7f660809dc20 2024-09-13T17:43:59.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660ce0f640 1 -- 172.21.15.96:0/2132348233 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66080712e0 con 0x7f660806f6d0 2024-09-13T17:43:59.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660ce0f640 1 --2- 172.21.15.96:0/2132348233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f6608134ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f65f80097f0 tx=0x7f65f801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fe2e8a634e20f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f66077fe640 1 -- 172.21.15.96:0/2132348233 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f80ab400 con 0x7f660806f6d0 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f66077fe640 1 -- 172.21.15.96:0/2132348233 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f80a7050 con 0x7f660806f6d0 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f66077fe640 1 -- 172.21.15.96:0/2132348233 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f80ab6b0 con 0x7f660806f6d0 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- 172.21.15.96:0/2132348233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 msgr2=0x7f6608134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 --2- 172.21.15.96:0/2132348233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f6608134ab0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f65f80097f0 tx=0x7f65f801c450 comp rx=0 tx=0).stop 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- 172.21.15.96:0/2132348233 shutdown_connections 2024-09-13T17:43:59.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 --2- 172.21.15.96:0/2132348233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f6608134ab0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- 172.21.15.96:0/2132348233 >> 172.21.15.96:0/2132348233 conn(0x7f6608098a60 msgr2=0x7f660809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- 172.21.15.96:0/2132348233 shutdown_connections 2024-09-13T17:43:59.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- 172.21.15.96:0/2132348233 wait complete. 2024-09-13T17:43:59.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 Processor -- start 2024-09-13T17:43:59.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.168+0000 7f660f073640 1 -- start start 2024-09-13T17:43:59.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660f073640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660f073640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66080a3ef0 con 0x7f660806f6d0 2024-09-13T17:43:59.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660ce0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660ce0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48954/0 (socket says 172.21.15.96:48954) 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660ce0f640 1 -- 172.21.15.96:0/727433309 learned_addr learned my addr 172.21.15.96:0/727433309 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660ce0f640 1 -- 172.21.15.96:0/727433309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80094d0 con 0x7f660806f6d0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660ce0f640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f65f8019060 tx=0x7f65f801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f8004020 con 0x7f660806f6d0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8004180 con 0x7f660806f6d0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66080a1de0 con 0x7f660806f6d0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f65f8004420 con 0x7f660806f6d0 2024-09-13T17:43:59.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66080a2ce0 con 0x7f660806f6d0 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f65f80b2040 con 0x7f660806f6d0 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66080a1490 con 0x7f660806f6d0 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 0x7f65f4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f65f80e20f0 con 0x7f660806f6d0 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f6607fff640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 0x7f65f4037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.172+0000 7f6607fff640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 0x7f65f4037a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000f70 tx=0x7f65fc007910 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.176+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f65f80c6710 con 0x7f660806f6d0 2024-09-13T17:43:59.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.352+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66080a51b0 con 0x7f660806f6d0 2024-09-13T17:43:59.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.352+0000 7f66057fa640 1 -- 172.21.15.96:0/727433309 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4427 (secure 0 0 0) 0x7f65f80b0090 con 0x7f660806f6d0 2024-09-13T17:43:59.358 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:59.358 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:43:59.358 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-09-13T17:43:59.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 msgr2=0x7f65f4037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 0x7f65f4037a80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f65fc000f70 tx=0x7f65fc007910 comp rx=0 tx=0).stop 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 msgr2=0x7f66080a3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f65f8019060 tx=0x7f65f801ac90 comp rx=0 tx=0).stop 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 shutdown_connections 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f65f40355c0 0x7f65f4037a80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 --2- 172.21.15.96:0/727433309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f660806f6d0 0x7f66080a3740 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 >> 172.21.15.96:0/727433309 conn(0x7f6608098a60 msgr2=0x7f660812fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 shutdown_connections 2024-09-13T17:43:59.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.356+0000 7f660f073640 1 -- 172.21.15.96:0/727433309 wait complete. 2024-09-13T17:43:59.376 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:43:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.484+0000 7f9dd58f8640 1 Processor -- start 2024-09-13T17:43:59.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.484+0000 7f9dd58f8640 1 -- start start 2024-09-13T17:43:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dd58f8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9dd009dc20 0x7f9dd01351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dd58f8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd006e130 con 0x7f9dd006f6d0 2024-09-13T17:43:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dd58f8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd00a6750 con 0x7f9dd009dc20 2024-09-13T17:43:59.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dce7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9dd009dc20 0x7f9dd01351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dceffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9dd006f6d0 0x7f9dd0134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57698/0 (socket says 172.21.15.96:57698) 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dce7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9dd009dc20 0x7f9dd01351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48966/0 (socket says 172.21.15.96:48966) 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dceffd640 1 -- 172.21.15.96:0/72857026 learned_addr learned my addr 172.21.15.96:0/72857026 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dce7fc640 1 -- 172.21.15.96:0/72857026 >> v1:172.21.15.96:6789/0 conn(0x7f9dd006f6d0 legacy=0x7f9dd0134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dce7fc640 1 -- 172.21.15.96:0/72857026 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00712e0 con 0x7f9dd009dc20 2024-09-13T17:43:59.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dcdffb640 1 -- 172.21.15.96:0/72857026 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685572797 0 0) 0x7f9dd006e130 con 0x7f9dd006f6d0 2024-09-13T17:43:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dce7fc640 1 --2- 172.21.15.96:0/72857026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd009dc20 0x7f9dd01351c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009740 tx=0x7f9dc401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf647d8bdf4e53b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dcdffb640 1 -- 172.21.15.96:0/72857026 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc40ab400 con 0x7f9dd009dc20 2024-09-13T17:43:59.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dcdffb640 1 -- 172.21.15.96:0/72857026 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc40a7050 con 0x7f9dd009dc20 2024-09-13T17:43:59.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dcdffb640 1 -- 172.21.15.96:0/72857026 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc40ab6b0 con 0x7f9dd009dc20 2024-09-13T17:43:59.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dd58f8640 1 -- 172.21.15.96:0/72857026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd009dc20 msgr2=0x7f9dd01351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.488+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/72857026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd009dc20 0x7f9dd01351c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009740 tx=0x7f9dc401c390 comp rx=0 tx=0).stop 2024-09-13T17:43:59.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/72857026 shutdown_connections 2024-09-13T17:43:59.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/72857026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd009dc20 0x7f9dd01351c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/72857026 >> 172.21.15.96:0/72857026 conn(0x7f9dd0098a60 msgr2=0x7f9dd009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/72857026 shutdown_connections 2024-09-13T17:43:59.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/72857026 wait complete. 2024-09-13T17:43:59.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 Processor -- start 2024-09-13T17:43:59.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- start start 2024-09-13T17:43:59.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd011cc00 con 0x7f9dd006f6d0 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48980/0 (socket says 172.21.15.96:48980) 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dceffd640 1 -- 172.21.15.96:0/56752423 learned_addr learned my addr 172.21.15.96:0/56752423 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dceffd640 1 -- 172.21.15.96:0/56752423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc4009420 con 0x7f9dd006f6d0 2024-09-13T17:43:59.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dceffd640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c420 tx=0x7f9dc000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc00074d0 con 0x7f9dd006f6d0 2024-09-13T17:43:59.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc0007630 con 0x7f9dd006f6d0 2024-09-13T17:43:59.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd00a59f0 con 0x7f9dd006f6d0 2024-09-13T17:43:59.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9dc0004960 con 0x7f9dd006f6d0 2024-09-13T17:43:59.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd0138f80 con 0x7f9dd006f6d0 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0136850 con 0x7f9dd006f6d0 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f9dc0004ac0 con 0x7f9dd006f6d0 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.492+0000 7f9db7fff640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 0x7f9dbc037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.496+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f9dc0047730 con 0x7f9dd006f6d0 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.496+0000 7f9dce7fc640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 0x7f9dbc037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.496+0000 7f9dce7fc640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 0x7f9dbc037a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ad000 tx=0x7f9dc4008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.500+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9dc001ab40 con 0x7f9dd006f6d0 2024-09-13T17:43:59.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.648+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9dd00a3b60 con 0x7f9dd006f6d0 2024-09-13T17:43:59.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.652+0000 7f9db7fff640 1 -- 172.21.15.96:0/56752423 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14765 (secure 0 0 0) 0x7f9dc0018070 con 0x7f9dd006f6d0 2024-09-13T17:43:59.657 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:59.657 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:57.009683+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 msgr2=0x7f9dbc037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 0x7f9dbc037a80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9dc40ad000 tx=0x7f9dc4008040 comp rx=0 tx=0).stop 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 msgr2=0x7f9dd01308e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000c420 tx=0x7f9dc000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 shutdown_connections 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f9dbc0355c0 0x7f9dbc037a80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 --2- 172.21.15.96:0/56752423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dd006f6d0 0x7f9dd01308e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 >> 172.21.15.96:0/56752423 conn(0x7f9dd0098a60 msgr2=0x7f9dd0133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 shutdown_connections 2024-09-13T17:43:59.660 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.656+0000 7f9dd58f8640 1 -- 172.21.15.96:0/56752423 wait complete. 2024-09-13T17:43:59.674 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:43:59.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:43:59.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfed2e5640 1 Processor -- start 2024-09-13T17:43:59.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfed2e5640 1 -- start start 2024-09-13T17:43:59.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfed2e5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbfe8134e70 0x7fbfe809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfed2e5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe809ed60 con 0x7fbfe8135250 2024-09-13T17:43:59.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfed2e5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfe809ddc0 con 0x7fbfe8134e70 2024-09-13T17:43:59.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfe6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbfe8134e70 0x7fbfe809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfe67fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbfe8135250 0x7fbfe8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57706/0 (socket says 172.21.15.96:57706) 2024-09-13T17:43:59.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfe67fc640 1 -- 172.21.15.96:0/1815951947 learned_addr learned my addr 172.21.15.96:0/1815951947 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:43:59.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfe6ffd640 1 -- 172.21.15.96:0/1815951947 >> v1:172.21.15.96:6789/0 conn(0x7fbfe8135250 legacy=0x7fbfe8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.776+0000 7fbfe6ffd640 1 -- 172.21.15.96:0/1815951947 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe80a68b0 con 0x7fbfe8134e70 2024-09-13T17:43:59.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfe6ffd640 1 --2- 172.21.15.96:0/1815951947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe809e820 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009d90 tx=0x7fbfe001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6354e289a6778661 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfe5ffb640 1 -- 172.21.15.96:0/1815951947 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe00a57b0 con 0x7fbfe8134e70 2024-09-13T17:43:59.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfe5ffb640 1 -- 172.21.15.96:0/1815951947 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00a5910 con 0x7fbfe8134e70 2024-09-13T17:43:59.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfe5ffb640 1 -- 172.21.15.96:0/1815951947 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe00a5bd0 con 0x7fbfe8134e70 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1815951947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 msgr2=0x7fbfe809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1815951947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe809e820 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0009d90 tx=0x7fbfe001c970 comp rx=0 tx=0).stop 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1815951947 shutdown_connections 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1815951947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe809e820 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1815951947 >> 172.21.15.96:0/1815951947 conn(0x7fbfe8098a60 msgr2=0x7fbfe809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1815951947 shutdown_connections 2024-09-13T17:43:59.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.780+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1815951947 wait complete. 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 Processor -- start 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 -- start start 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80a2f00 con 0x7fbfe8134e70 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48996/0 (socket says 172.21.15.96:48996) 2024-09-13T17:43:59.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe6ffd640 1 -- 172.21.15.96:0/1889434530 learned_addr learned my addr 172.21.15.96:0/1889434530 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe6ffd640 1 -- 172.21.15.96:0/1889434530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe0009a70 con 0x7fbfe8134e70 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe6ffd640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b0040 tx=0x7fbfe0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe00b9790 con 0x7fbfe8134e70 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe00b98f0 con 0x7fbfe8134e70 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe8132f50 con 0x7fbfe8134e70 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbfe00b9bd0 con 0x7fbfe8134e70 2024-09-13T17:43:59.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe80a1ff0 con 0x7fbfe8134e70 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fbfe00b9d30 con 0x7fbfe8134e70 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe81368b0 con 0x7fbfe8134e70 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 0x7fbfd4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fbfe00e33e0 con 0x7fbfe8134e70 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe67fc640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 0x7fbfd4037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:43:59.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.784+0000 7fbfe67fc640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 0x7fbfd4037a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc001610 tx=0x7fbfdc007ad0 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:43:59.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.788+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbfe00abd10 con 0x7fbfe8134e70 2024-09-13T17:43:59.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.928+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 --> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfe8136580 con 0x7fbfd40355c0 2024-09-13T17:43:59.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.964+0000 7fbfcf7fe640 1 -- 172.21.15.96:0/1889434530 <== mgr.4821 v2:172.21.15.96:6824/1425974344 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+486222 (secure 0 0 0) 0x7fbfdc001300 con 0x7fbfd40355c0 2024-09-13T17:43:59.972 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:43:59.978 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 msgr2=0x7fbfd4037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 0x7fbfd4037a80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc001610 tx=0x7fbfdc007ad0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 msgr2=0x7fbfe80a2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbfe00b0040 tx=0x7fbfe0019710 comp rx=0 tx=0).stop 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 shutdown_connections 2024-09-13T17:43:59.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fbfd40355c0 0x7fbfd4037a80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 --2- 172.21.15.96:0/1889434530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfe8134e70 0x7fbfe80a2740 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:43:59.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 >> 172.21.15.96:0/1889434530 conn(0x7fbfe8098a60 msgr2=0x7fbfe809abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:43:59.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 shutdown_connections 2024-09-13T17:43:59.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:43:59.972+0000 7fbfed2e5640 1 -- 172.21.15.96:0/1889434530 wait complete. 2024-09-13T17:43:59.991 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-13T17:43:59.602456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597724,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":292,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":341,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":91664,"kb_used_data":2392,"kb_used_omap":4,"kb_used_meta":89211,"kb_avail":283023856,"statfs":{"total":289910292480,"available":289816428544,"internally_reserved":0,"allocated":2449408,"data_stored":1630839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":91352403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":65,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.982939"},"pg_stats":[{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041642+0000","last_change":"2024-09-13T17:43:56.039642+0000","last_active":"2024-09-13T17:43:56.041642+0000","last_peered":"2024-09-13T17:43:56.041642+0000","last_clean":"2024-09-13T17:43:56.041642+0000","last_became_active":"2024-09-13T17:43:56.039348+0000","last_became_peered":"2024-09-13T17:43:56.039348+0000","last_unstale":"2024-09-13T17:43:56.041642+0000","last_undegraded":"2024-09-13T17:43:56.041642+0000","last_fullsized":"2024-09-13T17:43:56.041642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:06.416508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"36'4","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.028580+0000","last_change":"2024-09-13T17:43:36.597308+0000","last_active":"2024-09-13T17:43:56.028580+0000","last_peered":"2024-09-13T17:43:56.028580+0000","last_clean":"2024-09-13T17:43:56.028580+0000","last_became_active":"2024-09-13T17:43:36.596591+0000","last_became_peered":"2024-09-13T17:43:36.596591+0000","last_unstale":"2024-09-13T17:43:56.028580+0000","last_undegraded":"2024-09-13T17:43:56.028580+0000","last_fullsized":"2024-09-13T17:43:56.028580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:03.103037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652774+0000","last_change":"2024-09-13T17:43:36.997889+0000","last_active":"2024-09-13T17:43:55.652774+0000","last_peered":"2024-09-13T17:43:55.652774+0000","last_clean":"2024-09-13T17:43:55.652774+0000","last_became_active":"2024-09-13T17:43:36.997671+0000","last_became_peered":"2024-09-13T17:43:36.997671+0000","last_unstale":"2024-09-13T17:43:55.652774+0000","last_undegraded":"2024-09-13T17:43:55.652774+0000","last_fullsized":"2024-09-13T17:43:55.652774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018830+0000","last_change":"2024-09-13T17:43:55.672769+0000","last_active":"2024-09-13T17:43:57.018830+0000","last_peered":"2024-09-13T17:43:57.018830+0000","last_clean":"2024-09-13T17:43:57.018830+0000","last_became_active":"2024-09-13T17:43:55.672449+0000","last_became_peered":"2024-09-13T17:43:55.672449+0000","last_unstale":"2024-09-13T17:43:57.018830+0000","last_undegraded":"2024-09-13T17:43:57.018830+0000","last_fullsized":"2024-09-13T17:43:57.018830+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:58.723057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523341+0000","last_change":"2024-09-13T17:43:56.036340+0000","last_active":"2024-09-13T17:43:57.523341+0000","last_peered":"2024-09-13T17:43:57.523341+0000","last_clean":"2024-09-13T17:43:57.523341+0000","last_became_active":"2024-09-13T17:43:56.036034+0000","last_became_peered":"2024-09-13T17:43:56.036034+0000","last_unstale":"2024-09-13T17:43:57.523341+0000","last_undegraded":"2024-09-13T17:43:57.523341+0000","last_fullsized":"2024-09-13T17:43:57.523341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:43.427429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"36'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653024+0000","last_change":"2024-09-13T17:43:36.593051+0000","last_active":"2024-09-13T17:43:55.653024+0000","last_peered":"2024-09-13T17:43:55.653024+0000","last_clean":"2024-09-13T17:43:55.653024+0000","last_became_active":"2024-09-13T17:43:36.592210+0000","last_became_peered":"2024-09-13T17:43:36.592210+0000","last_unstale":"2024-09-13T17:43:55.653024+0000","last_undegraded":"2024-09-13T17:43:55.653024+0000","last_fullsized":"2024-09-13T17:43:55.653024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:07.519036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017125+0000","last_change":"2024-09-13T17:43:36.995868+0000","last_active":"2024-09-13T17:43:56.017125+0000","last_peered":"2024-09-13T17:43:56.017125+0000","last_clean":"2024-09-13T17:43:56.017125+0000","last_became_active":"2024-09-13T17:43:36.995304+0000","last_became_peered":"2024-09-13T17:43:36.995304+0000","last_unstale":"2024-09-13T17:43:56.017125+0000","last_undegraded":"2024-09-13T17:43:56.017125+0000","last_fullsized":"2024-09-13T17:43:56.017125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:40.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"41'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522247+0000","last_change":"2024-09-13T17:43:55.657425+0000","last_active":"2024-09-13T17:43:57.522247+0000","last_peered":"2024-09-13T17:43:57.522247+0000","last_clean":"2024-09-13T17:43:57.522247+0000","last_became_active":"2024-09-13T17:43:55.657150+0000","last_became_peered":"2024-09-13T17:43:55.657150+0000","last_unstale":"2024-09-13T17:43:57.522247+0000","last_undegraded":"2024-09-13T17:43:57.522247+0000","last_fullsized":"2024-09-13T17:43:57.522247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:52.217667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.056514+0000","last_change":"2024-09-13T17:43:56.055208+0000","last_active":"2024-09-13T17:43:56.056514+0000","last_peered":"2024-09-13T17:43:56.056514+0000","last_clean":"2024-09-13T17:43:56.056514+0000","last_became_active":"2024-09-13T17:43:56.054775+0000","last_became_peered":"2024-09-13T17:43:56.054775+0000","last_unstale":"2024-09-13T17:43:56.056514+0000","last_undegraded":"2024-09-13T17:43:56.056514+0000","last_fullsized":"2024-09-13T17:43:56.056514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:06.353192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652193+0000","last_change":"2024-09-13T17:43:36.607525+0000","last_active":"2024-09-13T17:43:55.652193+0000","last_peered":"2024-09-13T17:43:55.652193+0000","last_clean":"2024-09-13T17:43:55.652193+0000","last_became_active":"2024-09-13T17:43:36.606688+0000","last_became_peered":"2024-09-13T17:43:36.606688+0000","last_unstale":"2024-09-13T17:43:55.652193+0000","last_undegraded":"2024-09-13T17:43:55.652193+0000","last_fullsized":"2024-09-13T17:43:55.652193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:34.433256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024202+0000","last_change":"2024-09-13T17:43:36.994621+0000","last_active":"2024-09-13T17:43:56.024202+0000","last_peered":"2024-09-13T17:43:56.024202+0000","last_clean":"2024-09-13T17:43:56.024202+0000","last_became_active":"2024-09-13T17:43:36.993891+0000","last_became_peered":"2024-09-13T17:43:36.993891+0000","last_unstale":"2024-09-13T17:43:56.024202+0000","last_undegraded":"2024-09-13T17:43:56.024202+0000","last_fullsized":"2024-09-13T17:43:56.024202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"41'2","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.049310+0000","last_change":"2024-09-13T17:43:55.673793+0000","last_active":"2024-09-13T17:43:56.049310+0000","last_peered":"2024-09-13T17:43:56.049310+0000","last_clean":"2024-09-13T17:43:56.049310+0000","last_became_active":"2024-09-13T17:43:55.673508+0000","last_became_peered":"2024-09-13T17:43:55.673508+0000","last_unstale":"2024-09-13T17:43:56.049310+0000","last_undegraded":"2024-09-13T17:43:56.049310+0000","last_fullsized":"2024-09-13T17:43:56.049310+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:21.777386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019788+0000","last_change":"2024-09-13T17:43:56.046175+0000","last_active":"2024-09-13T17:43:57.019788+0000","last_peered":"2024-09-13T17:43:57.019788+0000","last_clean":"2024-09-13T17:43:57.019788+0000","last_became_active":"2024-09-13T17:43:56.044393+0000","last_became_peered":"2024-09-13T17:43:56.044393+0000","last_unstale":"2024-09-13T17:43:57.019788+0000","last_undegraded":"2024-09-13T17:43:57.019788+0000","last_fullsized":"2024-09-13T17:43:57.019788+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:29.059066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652772+0000","last_change":"2024-09-13T17:43:36.604117+0000","last_active":"2024-09-13T17:43:55.652772+0000","last_peered":"2024-09-13T17:43:55.652772+0000","last_clean":"2024-09-13T17:43:55.652772+0000","last_became_active":"2024-09-13T17:43:36.603754+0000","last_became_peered":"2024-09-13T17:43:36.603754+0000","last_unstale":"2024-09-13T17:43:55.652772+0000","last_undegraded":"2024-09-13T17:43:55.652772+0000","last_fullsized":"2024-09-13T17:43:55.652772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:48.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653156+0000","last_change":"2024-09-13T17:43:36.987727+0000","last_active":"2024-09-13T17:43:55.653156+0000","last_peered":"2024-09-13T17:43:55.653156+0000","last_clean":"2024-09-13T17:43:55.653156+0000","last_became_active":"2024-09-13T17:43:36.987360+0000","last_became_peered":"2024-09-13T17:43:36.987360+0000","last_unstale":"2024-09-13T17:43:55.653156+0000","last_undegraded":"2024-09-13T17:43:55.653156+0000","last_fullsized":"2024-09-13T17:43:55.653156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019087+0000","last_change":"2024-09-13T17:43:55.671195+0000","last_active":"2024-09-13T17:43:57.019087+0000","last_peered":"2024-09-13T17:43:57.019087+0000","last_clean":"2024-09-13T17:43:57.019087+0000","last_became_active":"2024-09-13T17:43:55.670558+0000","last_became_peered":"2024-09-13T17:43:55.670558+0000","last_unstale":"2024-09-13T17:43:57.019087+0000","last_undegraded":"2024-09-13T17:43:57.019087+0000","last_fullsized":"2024-09-13T17:43:57.019087+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:21.091910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527385+0000","last_change":"2024-09-13T17:43:56.040097+0000","last_active":"2024-09-13T17:43:57.527385+0000","last_peered":"2024-09-13T17:43:57.527385+0000","last_clean":"2024-09-13T17:43:57.527385+0000","last_became_active":"2024-09-13T17:43:56.039583+0000","last_became_peered":"2024-09-13T17:43:56.039583+0000","last_unstale":"2024-09-13T17:43:57.527385+0000","last_undegraded":"2024-09-13T17:43:57.527385+0000","last_fullsized":"2024-09-13T17:43:57.527385+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:02:17.654815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651196+0000","last_change":"2024-09-13T17:43:36.991670+0000","last_active":"2024-09-13T17:43:55.651196+0000","last_peered":"2024-09-13T17:43:55.651196+0000","last_clean":"2024-09-13T17:43:55.651196+0000","last_became_active":"2024-09-13T17:43:36.987763+0000","last_became_peered":"2024-09-13T17:43:36.987763+0000","last_unstale":"2024-09-13T17:43:55.651196+0000","last_undegraded":"2024-09-13T17:43:55.651196+0000","last_fullsized":"2024-09-13T17:43:55.651196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013749+0000","last_change":"2024-09-13T17:43:36.612138+0000","last_active":"2024-09-13T17:43:56.013749+0000","last_peered":"2024-09-13T17:43:56.013749+0000","last_clean":"2024-09-13T17:43:56.013749+0000","last_became_active":"2024-09-13T17:43:36.609873+0000","last_became_peered":"2024-09-13T17:43:36.609873+0000","last_unstale":"2024-09-13T17:43:56.013749+0000","last_undegraded":"2024-09-13T17:43:56.013749+0000","last_fullsized":"2024-09-13T17:43:56.013749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:49.578767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022188+0000","last_change":"2024-09-13T17:43:55.670827+0000","last_active":"2024-09-13T17:43:57.022188+0000","last_peered":"2024-09-13T17:43:57.022188+0000","last_clean":"2024-09-13T17:43:57.022188+0000","last_became_active":"2024-09-13T17:43:55.670513+0000","last_became_peered":"2024-09-13T17:43:55.670513+0000","last_unstale":"2024-09-13T17:43:57.022188+0000","last_undegraded":"2024-09-13T17:43:57.022188+0000","last_fullsized":"2024-09-13T17:43:57.022188+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:58.556852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045442+0000","last_change":"2024-09-13T17:43:56.040555+0000","last_active":"2024-09-13T17:43:56.045442+0000","last_peered":"2024-09-13T17:43:56.045442+0000","last_clean":"2024-09-13T17:43:56.045442+0000","last_became_active":"2024-09-13T17:43:56.040212+0000","last_became_peered":"2024-09-13T17:43:56.040212+0000","last_unstale":"2024-09-13T17:43:56.045442+0000","last_undegraded":"2024-09-13T17:43:56.045442+0000","last_fullsized":"2024-09-13T17:43:56.045442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:29.059066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022782+0000","last_change":"2024-09-13T17:43:36.977463+0000","last_active":"2024-09-13T17:43:56.022782+0000","last_peered":"2024-09-13T17:43:56.022782+0000","last_clean":"2024-09-13T17:43:56.022782+0000","last_became_active":"2024-09-13T17:43:36.977207+0000","last_became_peered":"2024-09-13T17:43:36.977207+0000","last_unstale":"2024-09-13T17:43:56.022782+0000","last_undegraded":"2024-09-13T17:43:56.022782+0000","last_fullsized":"2024-09-13T17:43:56.022782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:02:51.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651545+0000","last_change":"2024-09-13T17:43:36.607315+0000","last_active":"2024-09-13T17:43:55.651545+0000","last_peered":"2024-09-13T17:43:55.651545+0000","last_clean":"2024-09-13T17:43:55.651545+0000","last_became_active":"2024-09-13T17:43:36.606824+0000","last_became_peered":"2024-09-13T17:43:36.606824+0000","last_unstale":"2024-09-13T17:43:55.651545+0000","last_undegraded":"2024-09-13T17:43:55.651545+0000","last_fullsized":"2024-09-13T17:43:55.651545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.362407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024440+0000","last_change":"2024-09-13T17:43:55.674375+0000","last_active":"2024-09-13T17:43:56.024440+0000","last_peered":"2024-09-13T17:43:56.024440+0000","last_clean":"2024-09-13T17:43:56.024440+0000","last_became_active":"2024-09-13T17:43:55.673757+0000","last_became_peered":"2024-09-13T17:43:55.673757+0000","last_unstale":"2024-09-13T17:43:56.024440+0000","last_undegraded":"2024-09-13T17:43:56.024440+0000","last_fullsized":"2024-09-13T17:43:56.024440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:57.819380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522842+0000","last_change":"2024-09-13T17:43:56.045854+0000","last_active":"2024-09-13T17:43:57.522842+0000","last_peered":"2024-09-13T17:43:57.522842+0000","last_clean":"2024-09-13T17:43:57.522842+0000","last_became_active":"2024-09-13T17:43:56.041173+0000","last_became_peered":"2024-09-13T17:43:56.041173+0000","last_unstale":"2024-09-13T17:43:57.522842+0000","last_undegraded":"2024-09-13T17:43:57.522842+0000","last_fullsized":"2024-09-13T17:43:57.522842+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:00.766965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653358+0000","last_change":"2024-09-13T17:43:36.988413+0000","last_active":"2024-09-13T17:43:55.653358+0000","last_peered":"2024-09-13T17:43:55.653358+0000","last_clean":"2024-09-13T17:43:55.653358+0000","last_became_active":"2024-09-13T17:43:36.987084+0000","last_became_peered":"2024-09-13T17:43:36.987084+0000","last_unstale":"2024-09-13T17:43:55.653358+0000","last_undegraded":"2024-09-13T17:43:55.653358+0000","last_fullsized":"2024-09-13T17:43:55.653358+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"37'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653242+0000","last_change":"2024-09-13T17:43:36.596762+0000","last_active":"2024-09-13T17:43:55.653242+0000","last_peered":"2024-09-13T17:43:55.653242+0000","last_clean":"2024-09-13T17:43:55.653242+0000","last_became_active":"2024-09-13T17:43:36.594514+0000","last_became_peered":"2024-09-13T17:43:36.594514+0000","last_unstale":"2024-09-13T17:43:55.653242+0000","last_undegraded":"2024-09-13T17:43:55.653242+0000","last_fullsized":"2024-09-13T17:43:55.653242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:30:50.353395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017417+0000","last_change":"2024-09-13T17:43:55.666577+0000","last_active":"2024-09-13T17:43:56.017417+0000","last_peered":"2024-09-13T17:43:56.017417+0000","last_clean":"2024-09-13T17:43:56.017417+0000","last_became_active":"2024-09-13T17:43:55.666281+0000","last_became_peered":"2024-09-13T17:43:55.666281+0000","last_unstale":"2024-09-13T17:43:56.017417+0000","last_undegraded":"2024-09-13T17:43:56.017417+0000","last_fullsized":"2024-09-13T17:43:56.017417+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:58.723057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034241+0000","last_change":"2024-09-13T17:43:56.029459+0000","last_active":"2024-09-13T17:43:56.034241+0000","last_peered":"2024-09-13T17:43:56.034241+0000","last_clean":"2024-09-13T17:43:56.034241+0000","last_became_active":"2024-09-13T17:43:56.029291+0000","last_became_peered":"2024-09-13T17:43:56.029291+0000","last_unstale":"2024-09-13T17:43:56.034241+0000","last_undegraded":"2024-09-13T17:43:56.034241+0000","last_fullsized":"2024-09-13T17:43:56.034241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:07.604989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015004+0000","last_change":"2024-09-13T17:43:36.601888+0000","last_active":"2024-09-13T17:43:56.015004+0000","last_peered":"2024-09-13T17:43:56.015004+0000","last_clean":"2024-09-13T17:43:56.015004+0000","last_became_active":"2024-09-13T17:43:36.601588+0000","last_became_peered":"2024-09-13T17:43:36.601588+0000","last_unstale":"2024-09-13T17:43:56.015004+0000","last_undegraded":"2024-09-13T17:43:56.015004+0000","last_fullsized":"2024-09-13T17:43:56.015004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:05.159220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022938+0000","last_change":"2024-09-13T17:43:36.976896+0000","last_active":"2024-09-13T17:43:56.022938+0000","last_peered":"2024-09-13T17:43:56.022938+0000","last_clean":"2024-09-13T17:43:56.022938+0000","last_became_active":"2024-09-13T17:43:36.974032+0000","last_became_peered":"2024-09-13T17:43:36.974032+0000","last_unstale":"2024-09-13T17:43:56.022938+0000","last_undegraded":"2024-09-13T17:43:56.022938+0000","last_fullsized":"2024-09-13T17:43:56.022938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:52.403736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.15","version":"41'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020547+0000","last_change":"2024-09-13T17:43:55.674507+0000","last_active":"2024-09-13T17:43:57.020547+0000","last_peered":"2024-09-13T17:43:57.020547+0000","last_clean":"2024-09-13T17:43:57.020547+0000","last_became_active":"2024-09-13T17:43:55.674231+0000","last_became_peered":"2024-09-13T17:43:55.674231+0000","last_unstale":"2024-09-13T17:43:57.020547+0000","last_undegraded":"2024-09-13T17:43:57.020547+0000","last_fullsized":"2024-09-13T17:43:57.020547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:30.153604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.057152+0000","last_change":"2024-09-13T17:43:56.055279+0000","last_active":"2024-09-13T17:43:56.057152+0000","last_peered":"2024-09-13T17:43:56.057152+0000","last_clean":"2024-09-13T17:43:56.057152+0000","last_became_active":"2024-09-13T17:43:56.055033+0000","last_became_peered":"2024-09-13T17:43:56.055033+0000","last_unstale":"2024-09-13T17:43:56.057152+0000","last_undegraded":"2024-09-13T17:43:56.057152+0000","last_fullsized":"2024-09-13T17:43:56.057152+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:08.974623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014370+0000","last_change":"2024-09-13T17:43:36.999031+0000","last_active":"2024-09-13T17:43:56.014370+0000","last_peered":"2024-09-13T17:43:56.014370+0000","last_clean":"2024-09-13T17:43:56.014370+0000","last_became_active":"2024-09-13T17:43:36.998251+0000","last_became_peered":"2024-09-13T17:43:36.998251+0000","last_unstale":"2024-09-13T17:43:56.014370+0000","last_undegraded":"2024-09-13T17:43:56.014370+0000","last_fullsized":"2024-09-13T17:43:56.014370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:32.921571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015110+0000","last_change":"2024-09-13T17:43:36.612328+0000","last_active":"2024-09-13T17:43:56.015110+0000","last_peered":"2024-09-13T17:43:56.015110+0000","last_clean":"2024-09-13T17:43:56.015110+0000","last_became_active":"2024-09-13T17:43:36.610325+0000","last_became_peered":"2024-09-13T17:43:36.610325+0000","last_unstale":"2024-09-13T17:43:56.015110+0000","last_undegraded":"2024-09-13T17:43:56.015110+0000","last_fullsized":"2024-09-13T17:43:56.015110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:31.356170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523251+0000","last_change":"2024-09-13T17:43:55.663340+0000","last_active":"2024-09-13T17:43:57.523251+0000","last_peered":"2024-09-13T17:43:57.523251+0000","last_clean":"2024-09-13T17:43:57.523251+0000","last_became_active":"2024-09-13T17:43:55.660795+0000","last_became_peered":"2024-09-13T17:43:55.660795+0000","last_unstale":"2024-09-13T17:43:57.523251+0000","last_undegraded":"2024-09-13T17:43:57.523251+0000","last_fullsized":"2024-09-13T17:43:57.523251+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:23.130203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020905+0000","last_change":"2024-09-13T17:43:56.046809+0000","last_active":"2024-09-13T17:43:57.020905+0000","last_peered":"2024-09-13T17:43:57.020905+0000","last_clean":"2024-09-13T17:43:57.020905+0000","last_became_active":"2024-09-13T17:43:56.046375+0000","last_became_peered":"2024-09-13T17:43:56.046375+0000","last_unstale":"2024-09-13T17:43:57.020905+0000","last_undegraded":"2024-09-13T17:43:57.020905+0000","last_fullsized":"2024-09-13T17:43:57.020905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:08.974623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022281+0000","last_change":"2024-09-13T17:43:36.990056+0000","last_active":"2024-09-13T17:43:56.022281+0000","last_peered":"2024-09-13T17:43:56.022281+0000","last_clean":"2024-09-13T17:43:56.022281+0000","last_became_active":"2024-09-13T17:43:36.989640+0000","last_became_peered":"2024-09-13T17:43:36.989640+0000","last_unstale":"2024-09-13T17:43:56.022281+0000","last_undegraded":"2024-09-13T17:43:56.022281+0000","last_fullsized":"2024-09-13T17:43:56.022281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012938+0000","last_change":"2024-09-13T17:43:36.612085+0000","last_active":"2024-09-13T17:43:56.012938+0000","last_peered":"2024-09-13T17:43:56.012938+0000","last_clean":"2024-09-13T17:43:56.012938+0000","last_became_active":"2024-09-13T17:43:36.609135+0000","last_became_peered":"2024-09-13T17:43:36.609135+0000","last_unstale":"2024-09-13T17:43:56.012938+0000","last_undegraded":"2024-09-13T17:43:56.012938+0000","last_fullsized":"2024-09-13T17:43:56.012938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:22.346224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"41'2","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.047022+0000","last_change":"2024-09-13T17:43:55.673288+0000","last_active":"2024-09-13T17:43:56.047022+0000","last_peered":"2024-09-13T17:43:56.047022+0000","last_clean":"2024-09-13T17:43:56.047022+0000","last_became_active":"2024-09-13T17:43:55.672919+0000","last_became_peered":"2024-09-13T17:43:55.672919+0000","last_unstale":"2024-09-13T17:43:56.047022+0000","last_undegraded":"2024-09-13T17:43:56.047022+0000","last_fullsized":"2024-09-13T17:43:56.047022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:44.906793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524232+0000","last_change":"2024-09-13T17:43:56.053427+0000","last_active":"2024-09-13T17:43:57.524232+0000","last_peered":"2024-09-13T17:43:57.524232+0000","last_clean":"2024-09-13T17:43:57.524232+0000","last_became_active":"2024-09-13T17:43:56.051828+0000","last_became_peered":"2024-09-13T17:43:56.051828+0000","last_unstale":"2024-09-13T17:43:57.524232+0000","last_undegraded":"2024-09-13T17:43:57.524232+0000","last_fullsized":"2024-09-13T17:43:57.524232+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:34.985529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024628+0000","last_change":"2024-09-13T17:43:36.973857+0000","last_active":"2024-09-13T17:43:56.024628+0000","last_peered":"2024-09-13T17:43:56.024628+0000","last_clean":"2024-09-13T17:43:56.024628+0000","last_became_active":"2024-09-13T17:43:36.973595+0000","last_became_peered":"2024-09-13T17:43:36.973595+0000","last_unstale":"2024-09-13T17:43:56.024628+0000","last_undegraded":"2024-09-13T17:43:56.024628+0000","last_fullsized":"2024-09-13T17:43:56.024628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:06.165795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653699+0000","last_change":"2024-09-13T17:43:36.592618+0000","last_active":"2024-09-13T17:43:55.653699+0000","last_peered":"2024-09-13T17:43:55.653699+0000","last_clean":"2024-09-13T17:43:55.653699+0000","last_became_active":"2024-09-13T17:43:36.592303+0000","last_became_peered":"2024-09-13T17:43:36.592303+0000","last_unstale":"2024-09-13T17:43:55.653699+0000","last_undegraded":"2024-09-13T17:43:55.653699+0000","last_fullsized":"2024-09-13T17:43:55.653699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:16.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015889+0000","last_change":"2024-09-13T17:43:55.673141+0000","last_active":"2024-09-13T17:43:56.015889+0000","last_peered":"2024-09-13T17:43:56.015889+0000","last_clean":"2024-09-13T17:43:56.015889+0000","last_became_active":"2024-09-13T17:43:55.672869+0000","last_became_peered":"2024-09-13T17:43:55.672869+0000","last_unstale":"2024-09-13T17:43:56.015889+0000","last_undegraded":"2024-09-13T17:43:56.015889+0000","last_fullsized":"2024-09-13T17:43:56.015889+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:17.934538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526967+0000","last_change":"2024-09-13T17:43:56.037283+0000","last_active":"2024-09-13T17:43:57.526967+0000","last_peered":"2024-09-13T17:43:57.526967+0000","last_clean":"2024-09-13T17:43:57.526967+0000","last_became_active":"2024-09-13T17:43:56.036625+0000","last_became_peered":"2024-09-13T17:43:56.036625+0000","last_unstale":"2024-09-13T17:43:57.526967+0000","last_undegraded":"2024-09-13T17:43:57.526967+0000","last_fullsized":"2024-09-13T17:43:57.526967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:30.289577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017334+0000","last_change":"2024-09-13T17:43:36.991160+0000","last_active":"2024-09-13T17:43:56.017334+0000","last_peered":"2024-09-13T17:43:56.017334+0000","last_clean":"2024-09-13T17:43:56.017334+0000","last_became_active":"2024-09-13T17:43:36.990271+0000","last_became_peered":"2024-09-13T17:43:36.990271+0000","last_unstale":"2024-09-13T17:43:56.017334+0000","last_undegraded":"2024-09-13T17:43:56.017334+0000","last_fullsized":"2024-09-13T17:43:56.017334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:03.041558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013022+0000","last_change":"2024-09-13T17:43:36.612265+0000","last_active":"2024-09-13T17:43:56.013022+0000","last_peered":"2024-09-13T17:43:56.013022+0000","last_clean":"2024-09-13T17:43:56.013022+0000","last_became_active":"2024-09-13T17:43:36.609463+0000","last_became_peered":"2024-09-13T17:43:36.609463+0000","last_unstale":"2024-09-13T17:43:56.013022+0000","last_undegraded":"2024-09-13T17:43:56.013022+0000","last_fullsized":"2024-09-13T17:43:56.013022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:51.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.021956+0000","last_change":"2024-09-13T17:43:55.661979+0000","last_active":"2024-09-13T17:43:56.021956+0000","last_peered":"2024-09-13T17:43:56.021956+0000","last_clean":"2024-09-13T17:43:56.021956+0000","last_became_active":"2024-09-13T17:43:55.661618+0000","last_became_peered":"2024-09-13T17:43:55.661618+0000","last_unstale":"2024-09-13T17:43:56.021956+0000","last_undegraded":"2024-09-13T17:43:56.021956+0000","last_fullsized":"2024-09-13T17:43:56.021956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:55.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020077+0000","last_change":"2024-09-13T17:43:56.048692+0000","last_active":"2024-09-13T17:43:57.020077+0000","last_peered":"2024-09-13T17:43:57.020077+0000","last_clean":"2024-09-13T17:43:57.020077+0000","last_became_active":"2024-09-13T17:43:56.048428+0000","last_became_peered":"2024-09-13T17:43:56.048428+0000","last_unstale":"2024-09-13T17:43:57.020077+0000","last_undegraded":"2024-09-13T17:43:57.020077+0000","last_fullsized":"2024-09-13T17:43:57.020077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:06.416508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022630+0000","last_change":"2024-09-13T17:43:36.994616+0000","last_active":"2024-09-13T17:43:56.022630+0000","last_peered":"2024-09-13T17:43:56.022630+0000","last_clean":"2024-09-13T17:43:56.022630+0000","last_became_active":"2024-09-13T17:43:36.994356+0000","last_became_peered":"2024-09-13T17:43:36.994356+0000","last_unstale":"2024-09-13T17:43:56.022630+0000","last_undegraded":"2024-09-13T17:43:56.022630+0000","last_fullsized":"2024-09-13T17:43:56.022630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014245+0000","last_change":"2024-09-13T17:43:36.601819+0000","last_active":"2024-09-13T17:43:56.014245+0000","last_peered":"2024-09-13T17:43:56.014245+0000","last_clean":"2024-09-13T17:43:56.014245+0000","last_became_active":"2024-09-13T17:43:36.601091+0000","last_became_peered":"2024-09-13T17:43:36.601091+0000","last_unstale":"2024-09-13T17:43:56.014245+0000","last_undegraded":"2024-09-13T17:43:56.014245+0000","last_fullsized":"2024-09-13T17:43:56.014245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:22.130979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020772+0000","last_change":"2024-09-13T17:43:55.674696+0000","last_active":"2024-09-13T17:43:57.020772+0000","last_peered":"2024-09-13T17:43:57.020772+0000","last_clean":"2024-09-13T17:43:57.020772+0000","last_became_active":"2024-09-13T17:43:55.674339+0000","last_became_peered":"2024-09-13T17:43:55.674339+0000","last_unstale":"2024-09-13T17:43:57.020772+0000","last_undegraded":"2024-09-13T17:43:57.020772+0000","last_fullsized":"2024-09-13T17:43:57.020772+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:55.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527445+0000","last_change":"2024-09-13T17:43:56.040283+0000","last_active":"2024-09-13T17:43:57.527445+0000","last_peered":"2024-09-13T17:43:57.527445+0000","last_clean":"2024-09-13T17:43:57.527445+0000","last_became_active":"2024-09-13T17:43:56.039869+0000","last_became_peered":"2024-09-13T17:43:56.039869+0000","last_unstale":"2024-09-13T17:43:57.527445+0000","last_undegraded":"2024-09-13T17:43:57.527445+0000","last_fullsized":"2024-09-13T17:43:57.527445+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:18.322561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019552+0000","last_change":"2024-09-13T17:43:36.976449+0000","last_active":"2024-09-13T17:43:56.019552+0000","last_peered":"2024-09-13T17:43:56.019552+0000","last_clean":"2024-09-13T17:43:56.019552+0000","last_became_active":"2024-09-13T17:43:36.972460+0000","last_became_peered":"2024-09-13T17:43:36.972460+0000","last_unstale":"2024-09-13T17:43:56.019552+0000","last_undegraded":"2024-09-13T17:43:56.019552+0000","last_fullsized":"2024-09-13T17:43:56.019552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:12.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651120+0000","last_change":"2024-09-13T17:43:36.608221+0000","last_active":"2024-09-13T17:43:55.651120+0000","last_peered":"2024-09-13T17:43:55.651120+0000","last_clean":"2024-09-13T17:43:55.651120+0000","last_became_active":"2024-09-13T17:43:36.607779+0000","last_became_peered":"2024-09-13T17:43:36.607779+0000","last_unstale":"2024-09-13T17:43:55.651120+0000","last_undegraded":"2024-09-13T17:43:55.651120+0000","last_fullsized":"2024-09-13T17:43:55.651120+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:00.401036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526706+0000","last_change":"2024-09-13T17:43:55.676348+0000","last_active":"2024-09-13T17:43:57.526706+0000","last_peered":"2024-09-13T17:43:57.526706+0000","last_clean":"2024-09-13T17:43:57.526706+0000","last_became_active":"2024-09-13T17:43:55.675904+0000","last_became_peered":"2024-09-13T17:43:55.675904+0000","last_unstale":"2024-09-13T17:43:57.526706+0000","last_undegraded":"2024-09-13T17:43:57.526706+0000","last_fullsized":"2024-09-13T17:43:57.526706+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:31.669383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020695+0000","last_change":"2024-09-13T17:43:56.046636+0000","last_active":"2024-09-13T17:43:57.020695+0000","last_peered":"2024-09-13T17:43:57.020695+0000","last_clean":"2024-09-13T17:43:57.020695+0000","last_became_active":"2024-09-13T17:43:56.044857+0000","last_became_peered":"2024-09-13T17:43:56.044857+0000","last_unstale":"2024-09-13T17:43:57.020695+0000","last_undegraded":"2024-09-13T17:43:57.020695+0000","last_fullsized":"2024-09-13T17:43:57.020695+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.954905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.645714+0000","last_change":"2024-09-13T17:43:36.977858+0000","last_active":"2024-09-13T17:43:55.645714+0000","last_peered":"2024-09-13T17:43:55.645714+0000","last_clean":"2024-09-13T17:43:55.645714+0000","last_became_active":"2024-09-13T17:43:36.977282+0000","last_became_peered":"2024-09-13T17:43:36.977282+0000","last_unstale":"2024-09-13T17:43:55.645714+0000","last_undegraded":"2024-09-13T17:43:55.645714+0000","last_fullsized":"2024-09-13T17:43:55.645714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.642690+0000","last_change":"2024-09-13T17:43:36.601601+0000","last_active":"2024-09-13T17:43:55.642690+0000","last_peered":"2024-09-13T17:43:55.642690+0000","last_clean":"2024-09-13T17:43:55.642690+0000","last_became_active":"2024-09-13T17:43:36.594738+0000","last_became_peered":"2024-09-13T17:43:36.594738+0000","last_unstale":"2024-09-13T17:43:55.642690+0000","last_undegraded":"2024-09-13T17:43:55.642690+0000","last_fullsized":"2024-09-13T17:43:55.642690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:26.235235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527275+0000","last_change":"2024-09-13T17:43:55.676882+0000","last_active":"2024-09-13T17:43:57.527275+0000","last_peered":"2024-09-13T17:43:57.527275+0000","last_clean":"2024-09-13T17:43:57.527275+0000","last_became_active":"2024-09-13T17:43:55.676608+0000","last_became_peered":"2024-09-13T17:43:55.676608+0000","last_unstale":"2024-09-13T17:43:57.527275+0000","last_undegraded":"2024-09-13T17:43:57.527275+0000","last_fullsized":"2024-09-13T17:43:57.527275+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:05.943421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023194+0000","last_change":"2024-09-13T17:43:56.050364+0000","last_active":"2024-09-13T17:43:57.023194+0000","last_peered":"2024-09-13T17:43:57.023194+0000","last_clean":"2024-09-13T17:43:57.023194+0000","last_became_active":"2024-09-13T17:43:56.048480+0000","last_became_peered":"2024-09-13T17:43:56.048480+0000","last_unstale":"2024-09-13T17:43:57.023194+0000","last_undegraded":"2024-09-13T17:43:57.023194+0000","last_fullsized":"2024-09-13T17:43:57.023194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.761232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023521+0000","last_change":"2024-09-13T17:43:36.990024+0000","last_active":"2024-09-13T17:43:56.023521+0000","last_peered":"2024-09-13T17:43:56.023521+0000","last_clean":"2024-09-13T17:43:56.023521+0000","last_became_active":"2024-09-13T17:43:36.989774+0000","last_became_peered":"2024-09-13T17:43:36.989774+0000","last_unstale":"2024-09-13T17:43:56.023521+0000","last_undegraded":"2024-09-13T17:43:56.023521+0000","last_fullsized":"2024-09-13T17:43:56.023521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012637+0000","last_change":"2024-09-13T17:43:36.608307+0000","last_active":"2024-09-13T17:43:56.012637+0000","last_peered":"2024-09-13T17:43:56.012637+0000","last_clean":"2024-09-13T17:43:56.012637+0000","last_became_active":"2024-09-13T17:43:36.607846+0000","last_became_peered":"2024-09-13T17:43:36.607846+0000","last_unstale":"2024-09-13T17:43:56.012637+0000","last_undegraded":"2024-09-13T17:43:56.012637+0000","last_fullsized":"2024-09-13T17:43:56.012637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:34.779723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527037+0000","last_change":"2024-09-13T17:43:55.676200+0000","last_active":"2024-09-13T17:43:57.527037+0000","last_peered":"2024-09-13T17:43:57.527037+0000","last_clean":"2024-09-13T17:43:57.527037+0000","last_became_active":"2024-09-13T17:43:55.675799+0000","last_became_peered":"2024-09-13T17:43:55.675799+0000","last_unstale":"2024-09-13T17:43:57.527037+0000","last_undegraded":"2024-09-13T17:43:57.527037+0000","last_fullsized":"2024-09-13T17:43:57.527037+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:56.957972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023013+0000","last_change":"2024-09-13T17:43:56.045732+0000","last_active":"2024-09-13T17:43:57.023013+0000","last_peered":"2024-09-13T17:43:57.023013+0000","last_clean":"2024-09-13T17:43:57.023013+0000","last_became_active":"2024-09-13T17:43:56.045135+0000","last_became_peered":"2024-09-13T17:43:56.045135+0000","last_unstale":"2024-09-13T17:43:57.023013+0000","last_undegraded":"2024-09-13T17:43:57.023013+0000","last_fullsized":"2024-09-13T17:43:57.023013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:07.604989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013242+0000","last_change":"2024-09-13T17:43:36.999734+0000","last_active":"2024-09-13T17:43:56.013242+0000","last_peered":"2024-09-13T17:43:56.013242+0000","last_clean":"2024-09-13T17:43:56.013242+0000","last_became_active":"2024-09-13T17:43:36.999416+0000","last_became_peered":"2024-09-13T17:43:36.999416+0000","last_unstale":"2024-09-13T17:43:56.013242+0000","last_undegraded":"2024-09-13T17:43:56.013242+0000","last_fullsized":"2024-09-13T17:43:56.013242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:11.465274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.020946+0000","last_change":"2024-09-13T17:43:36.598625+0000","last_active":"2024-09-13T17:43:56.020946+0000","last_peered":"2024-09-13T17:43:56.020946+0000","last_clean":"2024-09-13T17:43:56.020946+0000","last_became_active":"2024-09-13T17:43:36.598229+0000","last_became_peered":"2024-09-13T17:43:36.598229+0000","last_unstale":"2024-09-13T17:43:56.020946+0000","last_undegraded":"2024-09-13T17:43:56.020946+0000","last_fullsized":"2024-09-13T17:43:56.020946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:54.846315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.529568+0000","last_change":"2024-09-13T17:43:55.676805+0000","last_active":"2024-09-13T17:43:57.529568+0000","last_peered":"2024-09-13T17:43:57.529568+0000","last_clean":"2024-09-13T17:43:57.529568+0000","last_became_active":"2024-09-13T17:43:55.676468+0000","last_became_peered":"2024-09-13T17:43:55.676468+0000","last_unstale":"2024-09-13T17:43:57.529568+0000","last_undegraded":"2024-09-13T17:43:57.529568+0000","last_fullsized":"2024-09-13T17:43:57.529568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:22.407289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.017942+0000","last_change":"2024-09-13T17:43:56.053288+0000","last_active":"2024-09-13T17:43:57.017942+0000","last_peered":"2024-09-13T17:43:57.017942+0000","last_clean":"2024-09-13T17:43:57.017942+0000","last_became_active":"2024-09-13T17:43:56.052585+0000","last_became_peered":"2024-09-13T17:43:56.052585+0000","last_unstale":"2024-09-13T17:43:57.017942+0000","last_undegraded":"2024-09-13T17:43:57.017942+0000","last_fullsized":"2024-09-13T17:43:57.017942+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:29.676452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015260+0000","last_change":"2024-09-13T17:43:36.972148+0000","last_active":"2024-09-13T17:43:56.015260+0000","last_peered":"2024-09-13T17:43:56.015260+0000","last_clean":"2024-09-13T17:43:56.015260+0000","last_became_active":"2024-09-13T17:43:36.971337+0000","last_became_peered":"2024-09-13T17:43:36.971337+0000","last_unstale":"2024-09-13T17:43:56.015260+0000","last_undegraded":"2024-09-13T17:43:56.015260+0000","last_fullsized":"2024-09-13T17:43:56.015260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:15.285058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"36'2","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014845+0000","last_change":"2024-09-13T17:43:36.601639+0000","last_active":"2024-09-13T17:43:56.014845+0000","last_peered":"2024-09-13T17:43:56.014845+0000","last_clean":"2024-09-13T17:43:56.014845+0000","last_became_active":"2024-09-13T17:43:36.601329+0000","last_became_peered":"2024-09-13T17:43:36.601329+0000","last_unstale":"2024-09-13T17:43:56.014845+0000","last_undegraded":"2024-09-13T17:43:56.014845+0000","last_fullsized":"2024-09-13T17:43:56.014845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:43.542629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"38'130","reported_seq":283,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017230+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:43:56.017230+0000","last_peered":"2024-09-13T17:43:56.017230+0000","last_clean":"2024-09-13T17:43:56.017230+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:43:56.017230+0000","last_undegraded":"2024-09-13T17:43:56.017230+0000","last_fullsized":"2024-09-13T17:43:56.017230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":197,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521921+0000","last_change":"2024-09-13T17:43:55.676840+0000","last_active":"2024-09-13T17:43:57.521921+0000","last_peered":"2024-09-13T17:43:57.521921+0000","last_clean":"2024-09-13T17:43:57.521921+0000","last_became_active":"2024-09-13T17:43:55.676501+0000","last_became_peered":"2024-09-13T17:43:55.676501+0000","last_unstale":"2024-09-13T17:43:57.521921+0000","last_undegraded":"2024-09-13T17:43:57.521921+0000","last_fullsized":"2024-09-13T17:43:57.521921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:52.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022963+0000","last_change":"2024-09-13T17:43:56.045890+0000","last_active":"2024-09-13T17:43:57.022963+0000","last_peered":"2024-09-13T17:43:57.022963+0000","last_clean":"2024-09-13T17:43:57.022963+0000","last_became_active":"2024-09-13T17:43:56.044886+0000","last_became_peered":"2024-09-13T17:43:56.044886+0000","last_unstale":"2024-09-13T17:43:57.022963+0000","last_undegraded":"2024-09-13T17:43:57.022963+0000","last_fullsized":"2024-09-13T17:43:57.022963+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:12.354007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.642583+0000","last_change":"2024-09-13T17:43:36.988557+0000","last_active":"2024-09-13T17:43:55.642583+0000","last_peered":"2024-09-13T17:43:55.642583+0000","last_clean":"2024-09-13T17:43:55.642583+0000","last_became_active":"2024-09-13T17:43:36.985705+0000","last_became_peered":"2024-09-13T17:43:36.985705+0000","last_unstale":"2024-09-13T17:43:55.642583+0000","last_undegraded":"2024-09-13T17:43:55.642583+0000","last_fullsized":"2024-09-13T17:43:55.642583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017258+0000","last_change":"2024-09-13T17:43:36.614029+0000","last_active":"2024-09-13T17:43:56.017258+0000","last_peered":"2024-09-13T17:43:56.017258+0000","last_clean":"2024-09-13T17:43:56.017258+0000","last_became_active":"2024-09-13T17:43:36.612938+0000","last_became_peered":"2024-09-13T17:43:36.612938+0000","last_unstale":"2024-09-13T17:43:56.017258+0000","last_undegraded":"2024-09-13T17:43:56.017258+0000","last_fullsized":"2024-09-13T17:43:56.017258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:57.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023168+0000","last_change":"2024-09-13T17:43:55.673716+0000","last_active":"2024-09-13T17:43:56.023168+0000","last_peered":"2024-09-13T17:43:56.023168+0000","last_clean":"2024-09-13T17:43:56.023168+0000","last_became_active":"2024-09-13T17:43:55.673446+0000","last_became_peered":"2024-09-13T17:43:55.673446+0000","last_unstale":"2024-09-13T17:43:56.023168+0000","last_undegraded":"2024-09-13T17:43:56.023168+0000","last_fullsized":"2024-09-13T17:43:56.023168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:50.027712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524051+0000","last_change":"2024-09-13T17:43:56.055697+0000","last_active":"2024-09-13T17:43:57.524051+0000","last_peered":"2024-09-13T17:43:57.524051+0000","last_clean":"2024-09-13T17:43:57.524051+0000","last_became_active":"2024-09-13T17:43:56.053781+0000","last_became_peered":"2024-09-13T17:43:56.053781+0000","last_unstale":"2024-09-13T17:43:57.524051+0000","last_undegraded":"2024-09-13T17:43:57.524051+0000","last_fullsized":"2024-09-13T17:43:57.524051+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:56.060802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652975+0000","last_change":"2024-09-13T17:43:36.987379+0000","last_active":"2024-09-13T17:43:55.652975+0000","last_peered":"2024-09-13T17:43:55.652975+0000","last_clean":"2024-09-13T17:43:55.652975+0000","last_became_active":"2024-09-13T17:43:36.986990+0000","last_became_peered":"2024-09-13T17:43:36.986990+0000","last_unstale":"2024-09-13T17:43:55.652975+0000","last_undegraded":"2024-09-13T17:43:55.652975+0000","last_fullsized":"2024-09-13T17:43:55.652975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652887+0000","last_change":"2024-09-13T17:43:36.596481+0000","last_active":"2024-09-13T17:43:55.652887+0000","last_peered":"2024-09-13T17:43:55.652887+0000","last_clean":"2024-09-13T17:43:55.652887+0000","last_became_active":"2024-09-13T17:43:36.594330+0000","last_became_peered":"2024-09-13T17:43:36.594330+0000","last_unstale":"2024-09-13T17:43:55.652887+0000","last_undegraded":"2024-09-13T17:43:55.652887+0000","last_fullsized":"2024-09-13T17:43:55.652887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:52.485090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019480+0000","last_change":"2024-09-13T17:43:55.671049+0000","last_active":"2024-09-13T17:43:57.019480+0000","last_peered":"2024-09-13T17:43:57.019480+0000","last_clean":"2024-09-13T17:43:57.019480+0000","last_became_active":"2024-09-13T17:43:55.670508+0000","last_became_peered":"2024-09-13T17:43:55.670508+0000","last_unstale":"2024-09-13T17:43:57.019480+0000","last_undegraded":"2024-09-13T17:43:57.019480+0000","last_fullsized":"2024-09-13T17:43:57.019480+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:30.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034358+0000","last_change":"2024-09-13T17:43:56.030662+0000","last_active":"2024-09-13T17:43:56.034358+0000","last_peered":"2024-09-13T17:43:56.034358+0000","last_clean":"2024-09-13T17:43:56.034358+0000","last_became_active":"2024-09-13T17:43:56.030409+0000","last_became_peered":"2024-09-13T17:43:56.030409+0000","last_unstale":"2024-09-13T17:43:56.034358+0000","last_undegraded":"2024-09-13T17:43:56.034358+0000","last_fullsized":"2024-09-13T17:43:56.034358+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:17.824302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026661+0000","last_change":"2024-09-13T17:43:36.994777+0000","last_active":"2024-09-13T17:43:56.026661+0000","last_peered":"2024-09-13T17:43:56.026661+0000","last_clean":"2024-09-13T17:43:56.026661+0000","last_became_active":"2024-09-13T17:43:36.994213+0000","last_became_peered":"2024-09-13T17:43:36.994213+0000","last_unstale":"2024-09-13T17:43:56.026661+0000","last_undegraded":"2024-09-13T17:43:56.026661+0000","last_fullsized":"2024-09-13T17:43:56.026661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"36'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652076+0000","last_change":"2024-09-13T17:43:36.607688+0000","last_active":"2024-09-13T17:43:55.652076+0000","last_peered":"2024-09-13T17:43:55.652076+0000","last_clean":"2024-09-13T17:43:55.652076+0000","last_became_active":"2024-09-13T17:43:36.607290+0000","last_became_peered":"2024-09-13T17:43:36.607290+0000","last_unstale":"2024-09-13T17:43:55.652076+0000","last_undegraded":"2024-09-13T17:43:55.652076+0000","last_fullsized":"2024-09-13T17:43:55.652076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:24.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015245+0000","last_change":"2024-09-13T17:43:55.666279+0000","last_active":"2024-09-13T17:43:56.015245+0000","last_peered":"2024-09-13T17:43:56.015245+0000","last_clean":"2024-09-13T17:43:56.015245+0000","last_became_active":"2024-09-13T17:43:55.666007+0000","last_became_peered":"2024-09-13T17:43:55.666007+0000","last_unstale":"2024-09-13T17:43:56.015245+0000","last_undegraded":"2024-09-13T17:43:56.015245+0000","last_fullsized":"2024-09-13T17:43:56.015245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:30.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522580+0000","last_change":"2024-09-13T17:43:56.036895+0000","last_active":"2024-09-13T17:43:57.522580+0000","last_peered":"2024-09-13T17:43:57.522580+0000","last_clean":"2024-09-13T17:43:57.522580+0000","last_became_active":"2024-09-13T17:43:56.035956+0000","last_became_peered":"2024-09-13T17:43:56.035956+0000","last_unstale":"2024-09-13T17:43:57.522580+0000","last_undegraded":"2024-09-13T17:43:57.522580+0000","last_fullsized":"2024-09-13T17:43:57.522580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:45:07.157910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017047+0000","last_change":"2024-09-13T17:43:36.996063+0000","last_active":"2024-09-13T17:43:56.017047+0000","last_peered":"2024-09-13T17:43:56.017047+0000","last_clean":"2024-09-13T17:43:56.017047+0000","last_became_active":"2024-09-13T17:43:36.995621+0000","last_became_peered":"2024-09-13T17:43:36.995621+0000","last_unstale":"2024-09-13T17:43:56.017047+0000","last_undegraded":"2024-09-13T17:43:56.017047+0000","last_fullsized":"2024-09-13T17:43:56.017047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:12.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653138+0000","last_change":"2024-09-13T17:43:36.594362+0000","last_active":"2024-09-13T17:43:55.653138+0000","last_peered":"2024-09-13T17:43:55.653138+0000","last_clean":"2024-09-13T17:43:55.653138+0000","last_became_active":"2024-09-13T17:43:36.593113+0000","last_became_peered":"2024-09-13T17:43:36.593113+0000","last_unstale":"2024-09-13T17:43:55.653138+0000","last_undegraded":"2024-09-13T17:43:55.653138+0000","last_fullsized":"2024-09-13T17:43:55.653138+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:01.736566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017493+0000","last_change":"2024-09-13T17:43:55.666777+0000","last_active":"2024-09-13T17:43:56.017493+0000","last_peered":"2024-09-13T17:43:56.017493+0000","last_clean":"2024-09-13T17:43:56.017493+0000","last_became_active":"2024-09-13T17:43:55.666063+0000","last_became_peered":"2024-09-13T17:43:55.666063+0000","last_unstale":"2024-09-13T17:43:56.017493+0000","last_undegraded":"2024-09-13T17:43:56.017493+0000","last_fullsized":"2024-09-13T17:43:56.017493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:14.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034293+0000","last_change":"2024-09-13T17:43:56.030128+0000","last_active":"2024-09-13T17:43:56.034293+0000","last_peered":"2024-09-13T17:43:56.034293+0000","last_clean":"2024-09-13T17:43:56.034293+0000","last_became_active":"2024-09-13T17:43:56.029794+0000","last_became_peered":"2024-09-13T17:43:56.029794+0000","last_unstale":"2024-09-13T17:43:56.034293+0000","last_undegraded":"2024-09-13T17:43:56.034293+0000","last_fullsized":"2024-09-13T17:43:56.034293+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:16:27.301097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026333+0000","last_change":"2024-09-13T17:43:36.996236+0000","last_active":"2024-09-13T17:43:56.026333+0000","last_peered":"2024-09-13T17:43:56.026333+0000","last_clean":"2024-09-13T17:43:56.026333+0000","last_became_active":"2024-09-13T17:43:36.995955+0000","last_became_peered":"2024-09-13T17:43:36.995955+0000","last_unstale":"2024-09-13T17:43:56.026333+0000","last_undegraded":"2024-09-13T17:43:56.026333+0000","last_fullsized":"2024-09-13T17:43:56.026333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:23.137553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.028670+0000","last_change":"2024-09-13T17:43:36.602229+0000","last_active":"2024-09-13T17:43:56.028670+0000","last_peered":"2024-09-13T17:43:56.028670+0000","last_clean":"2024-09-13T17:43:56.028670+0000","last_became_active":"2024-09-13T17:43:36.601122+0000","last_became_peered":"2024-09-13T17:43:36.601122+0000","last_unstale":"2024-09-13T17:43:56.028670+0000","last_undegraded":"2024-09-13T17:43:56.028670+0000","last_fullsized":"2024-09-13T17:43:56.028670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:25.802228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023043+0000","last_change":"2024-09-13T17:43:55.659694+0000","last_active":"2024-09-13T17:43:56.023043+0000","last_peered":"2024-09-13T17:43:56.023043+0000","last_clean":"2024-09-13T17:43:56.023043+0000","last_became_active":"2024-09-13T17:43:55.659358+0000","last_became_peered":"2024-09-13T17:43:55.659358+0000","last_unstale":"2024-09-13T17:43:56.023043+0000","last_undegraded":"2024-09-13T17:43:56.023043+0000","last_fullsized":"2024-09-13T17:43:56.023043+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:58.556852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.035445+0000","last_change":"2024-09-13T17:43:56.033294+0000","last_active":"2024-09-13T17:43:56.035445+0000","last_peered":"2024-09-13T17:43:56.035445+0000","last_clean":"2024-09-13T17:43:56.035445+0000","last_became_active":"2024-09-13T17:43:56.033034+0000","last_became_peered":"2024-09-13T17:43:56.033034+0000","last_unstale":"2024-09-13T17:43:56.035445+0000","last_undegraded":"2024-09-13T17:43:56.035445+0000","last_fullsized":"2024-09-13T17:43:56.035445+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:16.263521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"41'1","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.054080+0000","last_change":"2024-09-13T17:43:55.667221+0000","last_active":"2024-09-13T17:43:56.054080+0000","last_peered":"2024-09-13T17:43:56.054080+0000","last_clean":"2024-09-13T17:43:56.054080+0000","last_became_active":"2024-09-13T17:43:55.665780+0000","last_became_peered":"2024-09-13T17:43:55.665780+0000","last_unstale":"2024-09-13T17:43:56.054080+0000","last_undegraded":"2024-09-13T17:43:56.054080+0000","last_fullsized":"2024-09-13T17:43:56.054080+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:30.153604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023569+0000","last_change":"2024-09-13T17:43:36.967452+0000","last_active":"2024-09-13T17:43:56.023569+0000","last_peered":"2024-09-13T17:43:56.023569+0000","last_clean":"2024-09-13T17:43:56.023569+0000","last_became_active":"2024-09-13T17:43:36.966901+0000","last_became_peered":"2024-09-13T17:43:36.966901+0000","last_unstale":"2024-09-13T17:43:56.023569+0000","last_undegraded":"2024-09-13T17:43:56.023569+0000","last_fullsized":"2024-09-13T17:43:56.023569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"36'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012720+0000","last_change":"2024-09-13T17:43:36.607932+0000","last_active":"2024-09-13T17:43:56.012720+0000","last_peered":"2024-09-13T17:43:56.012720+0000","last_clean":"2024-09-13T17:43:56.012720+0000","last_became_active":"2024-09-13T17:43:36.607651+0000","last_became_peered":"2024-09-13T17:43:36.607651+0000","last_unstale":"2024-09-13T17:43:56.012720+0000","last_undegraded":"2024-09-13T17:43:56.012720+0000","last_fullsized":"2024-09-13T17:43:56.012720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:09.002010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526416+0000","last_change":"2024-09-13T17:43:56.046158+0000","last_active":"2024-09-13T17:43:57.526416+0000","last_peered":"2024-09-13T17:43:57.526416+0000","last_clean":"2024-09-13T17:43:57.526416+0000","last_became_active":"2024-09-13T17:43:56.041901+0000","last_became_peered":"2024-09-13T17:43:56.041901+0000","last_unstale":"2024-09-13T17:43:57.526416+0000","last_undegraded":"2024-09-13T17:43:57.526416+0000","last_fullsized":"2024-09-13T17:43:57.526416+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:40.596730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"41'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527303+0000","last_change":"2024-09-13T17:43:55.660775+0000","last_active":"2024-09-13T17:43:57.527303+0000","last_peered":"2024-09-13T17:43:57.527303+0000","last_clean":"2024-09-13T17:43:57.527303+0000","last_became_active":"2024-09-13T17:43:55.660354+0000","last_became_peered":"2024-09-13T17:43:55.660354+0000","last_unstale":"2024-09-13T17:43:57.527303+0000","last_undegraded":"2024-09-13T17:43:57.527303+0000","last_fullsized":"2024-09-13T17:43:57.527303+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:53.856576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.645660+0000","last_change":"2024-09-13T17:43:36.980350+0000","last_active":"2024-09-13T17:43:55.645660+0000","last_peered":"2024-09-13T17:43:55.645660+0000","last_clean":"2024-09-13T17:43:55.645660+0000","last_became_active":"2024-09-13T17:43:36.977778+0000","last_became_peered":"2024-09-13T17:43:36.977778+0000","last_unstale":"2024-09-13T17:43:55.645660+0000","last_undegraded":"2024-09-13T17:43:55.645660+0000","last_fullsized":"2024-09-13T17:43:55.645660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023368+0000","last_change":"2024-09-13T17:43:36.597400+0000","last_active":"2024-09-13T17:43:56.023368+0000","last_peered":"2024-09-13T17:43:56.023368+0000","last_clean":"2024-09-13T17:43:56.023368+0000","last_became_active":"2024-09-13T17:43:36.596736+0000","last_became_peered":"2024-09-13T17:43:36.596736+0000","last_unstale":"2024-09-13T17:43:56.023368+0000","last_undegraded":"2024-09-13T17:43:56.023368+0000","last_fullsized":"2024-09-13T17:43:56.023368+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:21.792293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523976+0000","last_change":"2024-09-13T17:43:56.040260+0000","last_active":"2024-09-13T17:43:57.523976+0000","last_peered":"2024-09-13T17:43:57.523976+0000","last_clean":"2024-09-13T17:43:57.523976+0000","last_became_active":"2024-09-13T17:43:56.039593+0000","last_became_peered":"2024-09-13T17:43:56.039593+0000","last_unstale":"2024-09-13T17:43:57.523976+0000","last_undegraded":"2024-09-13T17:43:57.523976+0000","last_fullsized":"2024-09-13T17:43:57.523976+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:11.552387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019628+0000","last_change":"2024-09-13T17:43:36.976611+0000","last_active":"2024-09-13T17:43:56.019628+0000","last_peered":"2024-09-13T17:43:56.019628+0000","last_clean":"2024-09-13T17:43:56.019628+0000","last_became_active":"2024-09-13T17:43:36.972871+0000","last_became_peered":"2024-09-13T17:43:36.972871+0000","last_unstale":"2024-09-13T17:43:56.019628+0000","last_undegraded":"2024-09-13T17:43:56.019628+0000","last_fullsized":"2024-09-13T17:43:56.019628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:02.818469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013857+0000","last_change":"2024-09-13T17:43:36.602194+0000","last_active":"2024-09-13T17:43:56.013857+0000","last_peered":"2024-09-13T17:43:56.013857+0000","last_clean":"2024-09-13T17:43:56.013857+0000","last_became_active":"2024-09-13T17:43:36.601939+0000","last_became_peered":"2024-09-13T17:43:36.601939+0000","last_unstale":"2024-09-13T17:43:56.013857+0000","last_undegraded":"2024-09-13T17:43:56.013857+0000","last_fullsized":"2024-09-13T17:43:56.013857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:32.706151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022308+0000","last_change":"2024-09-13T17:43:55.671328+0000","last_active":"2024-09-13T17:43:57.022308+0000","last_peered":"2024-09-13T17:43:57.022308+0000","last_clean":"2024-09-13T17:43:57.022308+0000","last_became_active":"2024-09-13T17:43:55.670859+0000","last_became_peered":"2024-09-13T17:43:55.670859+0000","last_unstale":"2024-09-13T17:43:57.022308+0000","last_undegraded":"2024-09-13T17:43:57.022308+0000","last_fullsized":"2024-09-13T17:43:57.022308+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:14.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526765+0000","last_change":"2024-09-13T17:43:56.041558+0000","last_active":"2024-09-13T17:43:57.526765+0000","last_peered":"2024-09-13T17:43:57.526765+0000","last_clean":"2024-09-13T17:43:57.526765+0000","last_became_active":"2024-09-13T17:43:56.041131+0000","last_became_peered":"2024-09-13T17:43:56.041131+0000","last_unstale":"2024-09-13T17:43:57.526765+0000","last_undegraded":"2024-09-13T17:43:57.526765+0000","last_fullsized":"2024-09-13T17:43:57.526765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:41.351412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651007+0000","last_change":"2024-09-13T17:43:36.988758+0000","last_active":"2024-09-13T17:43:55.651007+0000","last_peered":"2024-09-13T17:43:55.651007+0000","last_clean":"2024-09-13T17:43:55.651007+0000","last_became_active":"2024-09-13T17:43:36.987064+0000","last_became_peered":"2024-09-13T17:43:36.987064+0000","last_unstale":"2024-09-13T17:43:55.651007+0000","last_undegraded":"2024-09-13T17:43:55.651007+0000","last_fullsized":"2024-09-13T17:43:55.651007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"36'2","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.030955+0000","last_change":"2024-09-13T17:43:36.590347+0000","last_active":"2024-09-13T17:43:56.030955+0000","last_peered":"2024-09-13T17:43:56.030955+0000","last_clean":"2024-09-13T17:43:56.030955+0000","last_became_active":"2024-09-13T17:43:36.589982+0000","last_became_peered":"2024-09-13T17:43:36.589982+0000","last_unstale":"2024-09-13T17:43:56.030955+0000","last_undegraded":"2024-09-13T17:43:56.030955+0000","last_fullsized":"2024-09-13T17:43:56.030955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:03.593342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526922+0000","last_change":"2024-09-13T17:43:55.662879+0000","last_active":"2024-09-13T17:43:57.526922+0000","last_peered":"2024-09-13T17:43:57.526922+0000","last_clean":"2024-09-13T17:43:57.526922+0000","last_became_active":"2024-09-13T17:43:55.662566+0000","last_became_peered":"2024-09-13T17:43:55.662566+0000","last_unstale":"2024-09-13T17:43:57.526922+0000","last_undegraded":"2024-09-13T17:43:57.526922+0000","last_fullsized":"2024-09-13T17:43:57.526922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:45.840218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522284+0000","last_change":"2024-09-13T17:43:56.051810+0000","last_active":"2024-09-13T17:43:57.522284+0000","last_peered":"2024-09-13T17:43:57.522284+0000","last_clean":"2024-09-13T17:43:57.522284+0000","last_became_active":"2024-09-13T17:43:56.051201+0000","last_became_peered":"2024-09-13T17:43:56.051201+0000","last_unstale":"2024-09-13T17:43:57.522284+0000","last_undegraded":"2024-09-13T17:43:57.522284+0000","last_fullsized":"2024-09-13T17:43:57.522284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:15.865568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024246+0000","last_change":"2024-09-13T17:43:36.991166+0000","last_active":"2024-09-13T17:43:56.024246+0000","last_peered":"2024-09-13T17:43:56.024246+0000","last_clean":"2024-09-13T17:43:56.024246+0000","last_became_active":"2024-09-13T17:43:36.990864+0000","last_became_peered":"2024-09-13T17:43:36.990864+0000","last_unstale":"2024-09-13T17:43:56.024246+0000","last_undegraded":"2024-09-13T17:43:56.024246+0000","last_fullsized":"2024-09-13T17:43:56.024246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"38'11","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023871+0000","last_change":"2024-09-13T17:43:36.603712+0000","last_active":"2024-09-13T17:43:56.023871+0000","last_peered":"2024-09-13T17:43:56.023871+0000","last_clean":"2024-09-13T17:43:56.023871+0000","last_became_active":"2024-09-13T17:43:36.603454+0000","last_became_peered":"2024-09-13T17:43:36.603454+0000","last_unstale":"2024-09-13T17:43:56.023871+0000","last_undegraded":"2024-09-13T17:43:56.023871+0000","last_fullsized":"2024-09-13T17:43:56.023871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:01.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523554+0000","last_change":"2024-09-13T17:43:55.676053+0000","last_active":"2024-09-13T17:43:57.523554+0000","last_peered":"2024-09-13T17:43:57.523554+0000","last_clean":"2024-09-13T17:43:57.523554+0000","last_became_active":"2024-09-13T17:43:55.675758+0000","last_became_peered":"2024-09-13T17:43:55.675758+0000","last_unstale":"2024-09-13T17:43:57.523554+0000","last_undegraded":"2024-09-13T17:43:57.523554+0000","last_fullsized":"2024-09-13T17:43:57.523554+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:37.937247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034212+0000","last_change":"2024-09-13T17:43:56.028847+0000","last_active":"2024-09-13T17:43:56.034212+0000","last_peered":"2024-09-13T17:43:56.034212+0000","last_clean":"2024-09-13T17:43:56.034212+0000","last_became_active":"2024-09-13T17:43:56.028496+0000","last_became_peered":"2024-09-13T17:43:56.028496+0000","last_unstale":"2024-09-13T17:43:56.034212+0000","last_undegraded":"2024-09-13T17:43:56.034212+0000","last_fullsized":"2024-09-13T17:43:56.034212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.090860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652649+0000","last_change":"2024-09-13T17:43:36.988822+0000","last_active":"2024-09-13T17:43:55.652649+0000","last_peered":"2024-09-13T17:43:55.652649+0000","last_clean":"2024-09-13T17:43:55.652649+0000","last_became_active":"2024-09-13T17:43:36.987189+0000","last_became_peered":"2024-09-13T17:43:36.987189+0000","last_unstale":"2024-09-13T17:43:55.652649+0000","last_undegraded":"2024-09-13T17:43:55.652649+0000","last_fullsized":"2024-09-13T17:43:55.652649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"36'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024766+0000","last_change":"2024-09-13T17:43:36.604426+0000","last_active":"2024-09-13T17:43:56.024766+0000","last_peered":"2024-09-13T17:43:56.024766+0000","last_clean":"2024-09-13T17:43:56.024766+0000","last_became_active":"2024-09-13T17:43:36.603659+0000","last_became_peered":"2024-09-13T17:43:36.603659+0000","last_unstale":"2024-09-13T17:43:56.024766+0000","last_undegraded":"2024-09-13T17:43:56.024766+0000","last_fullsized":"2024-09-13T17:43:56.024766+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:27.409650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020106+0000","last_change":"2024-09-13T17:43:55.671013+0000","last_active":"2024-09-13T17:43:57.020106+0000","last_peered":"2024-09-13T17:43:57.020106+0000","last_clean":"2024-09-13T17:43:57.020106+0000","last_became_active":"2024-09-13T17:43:55.670751+0000","last_became_peered":"2024-09-13T17:43:55.670751+0000","last_unstale":"2024-09-13T17:43:57.020106+0000","last_undegraded":"2024-09-13T17:43:57.020106+0000","last_fullsized":"2024-09-13T17:43:57.020106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:50.971922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019010+0000","last_change":"2024-09-13T17:43:56.048672+0000","last_active":"2024-09-13T17:43:57.019010+0000","last_peered":"2024-09-13T17:43:57.019010+0000","last_clean":"2024-09-13T17:43:57.019010+0000","last_became_active":"2024-09-13T17:43:56.048212+0000","last_became_peered":"2024-09-13T17:43:56.048212+0000","last_unstale":"2024-09-13T17:43:57.019010+0000","last_undegraded":"2024-09-13T17:43:57.019010+0000","last_fullsized":"2024-09-13T17:43:57.019010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:04.391077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653121+0000","last_change":"2024-09-13T17:43:36.989522+0000","last_active":"2024-09-13T17:43:55.653121+0000","last_peered":"2024-09-13T17:43:55.653121+0000","last_clean":"2024-09-13T17:43:55.653121+0000","last_became_active":"2024-09-13T17:43:36.987659+0000","last_became_peered":"2024-09-13T17:43:36.987659+0000","last_unstale":"2024-09-13T17:43:55.653121+0000","last_undegraded":"2024-09-13T17:43:55.653121+0000","last_fullsized":"2024-09-13T17:43:55.653121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014845+0000","last_change":"2024-09-13T17:43:36.610299+0000","last_active":"2024-09-13T17:43:56.014845+0000","last_peered":"2024-09-13T17:43:56.014845+0000","last_clean":"2024-09-13T17:43:56.014845+0000","last_became_active":"2024-09-13T17:43:36.609995+0000","last_became_peered":"2024-09-13T17:43:36.609995+0000","last_unstale":"2024-09-13T17:43:56.014845+0000","last_undegraded":"2024-09-13T17:43:56.014845+0000","last_fullsized":"2024-09-13T17:43:56.014845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:06.509485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522006+0000","last_change":"2024-09-13T17:43:55.660304+0000","last_active":"2024-09-13T17:43:57.522006+0000","last_peered":"2024-09-13T17:43:57.522006+0000","last_clean":"2024-09-13T17:43:57.522006+0000","last_became_active":"2024-09-13T17:43:55.658864+0000","last_became_peered":"2024-09-13T17:43:55.658864+0000","last_unstale":"2024-09-13T17:43:57.522006+0000","last_undegraded":"2024-09-13T17:43:57.522006+0000","last_fullsized":"2024-09-13T17:43:57.522006+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:53.802466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018659+0000","last_change":"2024-09-13T17:43:56.053153+0000","last_active":"2024-09-13T17:43:57.018659+0000","last_peered":"2024-09-13T17:43:57.018659+0000","last_clean":"2024-09-13T17:43:57.018659+0000","last_became_active":"2024-09-13T17:43:56.052913+0000","last_became_peered":"2024-09-13T17:43:56.052913+0000","last_unstale":"2024-09-13T17:43:57.018659+0000","last_undegraded":"2024-09-13T17:43:57.018659+0000","last_fullsized":"2024-09-13T17:43:57.018659+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:03:00.387308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652939+0000","last_change":"2024-09-13T17:43:36.994355+0000","last_active":"2024-09-13T17:43:55.652939+0000","last_peered":"2024-09-13T17:43:55.652939+0000","last_clean":"2024-09-13T17:43:55.652939+0000","last_became_active":"2024-09-13T17:43:36.988868+0000","last_became_peered":"2024-09-13T17:43:36.988868+0000","last_unstale":"2024-09-13T17:43:55.652939+0000","last_undegraded":"2024-09-13T17:43:55.652939+0000","last_fullsized":"2024-09-13T17:43:55.652939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653442+0000","last_change":"2024-09-13T17:43:36.606733+0000","last_active":"2024-09-13T17:43:55.653442+0000","last_peered":"2024-09-13T17:43:55.653442+0000","last_clean":"2024-09-13T17:43:55.653442+0000","last_became_active":"2024-09-13T17:43:36.604421+0000","last_became_peered":"2024-09-13T17:43:36.604421+0000","last_unstale":"2024-09-13T17:43:55.653442+0000","last_undegraded":"2024-09-13T17:43:55.653442+0000","last_fullsized":"2024-09-13T17:43:55.653442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:21.923719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022946+0000","last_change":"2024-09-13T17:43:55.662182+0000","last_active":"2024-09-13T17:43:56.022946+0000","last_peered":"2024-09-13T17:43:56.022946+0000","last_clean":"2024-09-13T17:43:56.022946+0000","last_became_active":"2024-09-13T17:43:55.661810+0000","last_became_peered":"2024-09-13T17:43:55.661810+0000","last_unstale":"2024-09-13T17:43:56.022946+0000","last_undegraded":"2024-09-13T17:43:56.022946+0000","last_fullsized":"2024-09-13T17:43:56.022946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:21.091910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045655+0000","last_change":"2024-09-13T17:43:56.042895+0000","last_active":"2024-09-13T17:43:56.045655+0000","last_peered":"2024-09-13T17:43:56.045655+0000","last_clean":"2024-09-13T17:43:56.045655+0000","last_became_active":"2024-09-13T17:43:56.042430+0000","last_became_peered":"2024-09-13T17:43:56.042430+0000","last_unstale":"2024-09-13T17:43:56.045655+0000","last_undegraded":"2024-09-13T17:43:56.045655+0000","last_fullsized":"2024-09-13T17:43:56.045655+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.432317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013956+0000","last_change":"2024-09-13T17:43:36.612531+0000","last_active":"2024-09-13T17:43:56.013956+0000","last_peered":"2024-09-13T17:43:56.013956+0000","last_clean":"2024-09-13T17:43:56.013956+0000","last_became_active":"2024-09-13T17:43:36.610289+0000","last_became_peered":"2024-09-13T17:43:36.610289+0000","last_unstale":"2024-09-13T17:43:56.013956+0000","last_undegraded":"2024-09-13T17:43:56.013956+0000","last_fullsized":"2024-09-13T17:43:56.013956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:09.724844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019699+0000","last_change":"2024-09-13T17:43:36.976787+0000","last_active":"2024-09-13T17:43:56.019699+0000","last_peered":"2024-09-13T17:43:56.019699+0000","last_clean":"2024-09-13T17:43:56.019699+0000","last_became_active":"2024-09-13T17:43:36.976205+0000","last_became_peered":"2024-09-13T17:43:36.976205+0000","last_unstale":"2024-09-13T17:43:56.019699+0000","last_undegraded":"2024-09-13T17:43:56.019699+0000","last_fullsized":"2024-09-13T17:43:56.019699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:09.035544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"41'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524234+0000","last_change":"2024-09-13T17:43:55.676700+0000","last_active":"2024-09-13T17:43:57.524234+0000","last_peered":"2024-09-13T17:43:57.524234+0000","last_clean":"2024-09-13T17:43:57.524234+0000","last_became_active":"2024-09-13T17:43:55.676075+0000","last_became_peered":"2024-09-13T17:43:55.676075+0000","last_unstale":"2024-09-13T17:43:57.524234+0000","last_undegraded":"2024-09-13T17:43:57.524234+0000","last_fullsized":"2024-09-13T17:43:57.524234+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:59.379056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020569+0000","last_change":"2024-09-13T17:43:56.043210+0000","last_active":"2024-09-13T17:43:57.020569+0000","last_peered":"2024-09-13T17:43:57.020569+0000","last_clean":"2024-09-13T17:43:57.020569+0000","last_became_active":"2024-09-13T17:43:56.042583+0000","last_became_peered":"2024-09-13T17:43:56.042583+0000","last_unstale":"2024-09-13T17:43:57.020569+0000","last_undegraded":"2024-09-13T17:43:57.020569+0000","last_fullsized":"2024-09-13T17:43:57.020569+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:06.353192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022313+0000","last_change":"2024-09-13T17:43:36.990968+0000","last_active":"2024-09-13T17:43:56.022313+0000","last_peered":"2024-09-13T17:43:56.022313+0000","last_clean":"2024-09-13T17:43:56.022313+0000","last_became_active":"2024-09-13T17:43:36.990688+0000","last_became_peered":"2024-09-13T17:43:36.990688+0000","last_unstale":"2024-09-13T17:43:56.022313+0000","last_undegraded":"2024-09-13T17:43:56.022313+0000","last_fullsized":"2024-09-13T17:43:56.022313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022735+0000","last_change":"2024-09-13T17:43:56.048225+0000","last_active":"2024-09-13T17:43:57.022735+0000","last_peered":"2024-09-13T17:43:57.022735+0000","last_clean":"2024-09-13T17:43:57.022735+0000","last_became_active":"2024-09-13T17:43:56.047392+0000","last_became_peered":"2024-09-13T17:43:56.047392+0000","last_unstale":"2024-09-13T17:43:57.022735+0000","last_undegraded":"2024-09-13T17:43:57.022735+0000","last_fullsized":"2024-09-13T17:43:57.022735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:23.415541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.638887+0000","last_change":"2024-09-13T17:43:36.989315+0000","last_active":"2024-09-13T17:43:55.638887+0000","last_peered":"2024-09-13T17:43:55.638887+0000","last_clean":"2024-09-13T17:43:55.638887+0000","last_became_active":"2024-09-13T17:43:36.989070+0000","last_became_peered":"2024-09-13T17:43:36.989070+0000","last_unstale":"2024-09-13T17:43:55.638887+0000","last_undegraded":"2024-09-13T17:43:55.638887+0000","last_fullsized":"2024-09-13T17:43:55.638887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.043071+0000","last_change":"2024-09-13T17:43:56.040467+0000","last_active":"2024-09-13T17:43:56.043071+0000","last_peered":"2024-09-13T17:43:56.043071+0000","last_clean":"2024-09-13T17:43:56.043071+0000","last_became_active":"2024-09-13T17:43:56.039885+0000","last_became_peered":"2024-09-13T17:43:56.039885+0000","last_unstale":"2024-09-13T17:43:56.043071+0000","last_undegraded":"2024-09-13T17:43:56.043071+0000","last_fullsized":"2024-09-13T17:43:56.043071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.954905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024569+0000","last_change":"2024-09-13T17:43:36.967204+0000","last_active":"2024-09-13T17:43:56.024569+0000","last_peered":"2024-09-13T17:43:56.024569+0000","last_clean":"2024-09-13T17:43:56.024569+0000","last_became_active":"2024-09-13T17:43:36.966928+0000","last_became_peered":"2024-09-13T17:43:36.966928+0000","last_unstale":"2024-09-13T17:43:56.024569+0000","last_undegraded":"2024-09-13T17:43:56.024569+0000","last_fullsized":"2024-09-13T17:43:56.024569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:58.179836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020461+0000","last_change":"2024-09-13T17:43:56.043877+0000","last_active":"2024-09-13T17:43:57.020461+0000","last_peered":"2024-09-13T17:43:57.020461+0000","last_clean":"2024-09-13T17:43:57.020461+0000","last_became_active":"2024-09-13T17:43:56.043563+0000","last_became_peered":"2024-09-13T17:43:56.043563+0000","last_unstale":"2024-09-13T17:43:57.020461+0000","last_undegraded":"2024-09-13T17:43:57.020461+0000","last_fullsized":"2024-09-13T17:43:57.020461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:39.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019774+0000","last_change":"2024-09-13T17:43:36.998865+0000","last_active":"2024-09-13T17:43:56.019774+0000","last_peered":"2024-09-13T17:43:56.019774+0000","last_clean":"2024-09-13T17:43:56.019774+0000","last_became_active":"2024-09-13T17:43:36.998611+0000","last_became_peered":"2024-09-13T17:43:36.998611+0000","last_unstale":"2024-09-13T17:43:56.019774+0000","last_undegraded":"2024-09-13T17:43:56.019774+0000","last_fullsized":"2024-09-13T17:43:56.019774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:31.977360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019616+0000","last_change":"2024-09-13T17:43:56.056341+0000","last_active":"2024-09-13T17:43:57.019616+0000","last_peered":"2024-09-13T17:43:57.019616+0000","last_clean":"2024-09-13T17:43:57.019616+0000","last_became_active":"2024-09-13T17:43:56.056095+0000","last_became_peered":"2024-09-13T17:43:56.056095+0000","last_unstale":"2024-09-13T17:43:57.019616+0000","last_undegraded":"2024-09-13T17:43:57.019616+0000","last_fullsized":"2024-09-13T17:43:57.019616+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:07.599159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024284+0000","last_change":"2024-09-13T17:43:36.994453+0000","last_active":"2024-09-13T17:43:56.024284+0000","last_peered":"2024-09-13T17:43:56.024284+0000","last_clean":"2024-09-13T17:43:56.024284+0000","last_became_active":"2024-09-13T17:43:36.993574+0000","last_became_peered":"2024-09-13T17:43:36.993574+0000","last_unstale":"2024-09-13T17:43:56.024284+0000","last_undegraded":"2024-09-13T17:43:56.024284+0000","last_fullsized":"2024-09-13T17:43:56.024284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521775+0000","last_change":"2024-09-13T17:43:56.051980+0000","last_active":"2024-09-13T17:43:57.521775+0000","last_peered":"2024-09-13T17:43:57.521775+0000","last_clean":"2024-09-13T17:43:57.521775+0000","last_became_active":"2024-09-13T17:43:56.051475+0000","last_became_peered":"2024-09-13T17:43:56.051475+0000","last_unstale":"2024-09-13T17:43:57.521775+0000","last_undegraded":"2024-09-13T17:43:57.521775+0000","last_fullsized":"2024-09-13T17:43:57.521775+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:40:24.707236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015450+0000","last_change":"2024-09-13T17:43:36.999236+0000","last_active":"2024-09-13T17:43:56.015450+0000","last_peered":"2024-09-13T17:43:56.015450+0000","last_clean":"2024-09-13T17:43:56.015450+0000","last_became_active":"2024-09-13T17:43:36.998617+0000","last_became_peered":"2024-09-13T17:43:36.998617+0000","last_unstale":"2024-09-13T17:43:56.015450+0000","last_undegraded":"2024-09-13T17:43:56.015450+0000","last_fullsized":"2024-09-13T17:43:56.015450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:43.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.042763+0000","last_change":"2024-09-13T17:43:56.039906+0000","last_active":"2024-09-13T17:43:56.042763+0000","last_peered":"2024-09-13T17:43:56.042763+0000","last_clean":"2024-09-13T17:43:56.042763+0000","last_became_active":"2024-09-13T17:43:56.039098+0000","last_became_peered":"2024-09-13T17:43:56.039098+0000","last_unstale":"2024-09-13T17:43:56.042763+0000","last_undegraded":"2024-09-13T17:43:56.042763+0000","last_fullsized":"2024-09-13T17:43:56.042763+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:33.996348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026414+0000","last_change":"2024-09-13T17:43:36.995992+0000","last_active":"2024-09-13T17:43:56.026414+0000","last_peered":"2024-09-13T17:43:56.026414+0000","last_clean":"2024-09-13T17:43:56.026414+0000","last_became_active":"2024-09-13T17:43:36.995839+0000","last_became_peered":"2024-09-13T17:43:36.995839+0000","last_unstale":"2024-09-13T17:43:56.026414+0000","last_undegraded":"2024-09-13T17:43:56.026414+0000","last_fullsized":"2024-09-13T17:43:56.026414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:43.285004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041609+0000","last_change":"2024-09-13T17:43:56.039634+0000","last_active":"2024-09-13T17:43:56.041609+0000","last_peered":"2024-09-13T17:43:56.041609+0000","last_clean":"2024-09-13T17:43:56.041609+0000","last_became_active":"2024-09-13T17:43:56.039393+0000","last_became_peered":"2024-09-13T17:43:56.039393+0000","last_unstale":"2024-09-13T17:43:56.041609+0000","last_undegraded":"2024-09-13T17:43:56.041609+0000","last_fullsized":"2024-09-13T17:43:56.041609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:02.407638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017201+0000","last_change":"2024-09-13T17:43:36.972052+0000","last_active":"2024-09-13T17:43:56.017201+0000","last_peered":"2024-09-13T17:43:56.017201+0000","last_clean":"2024-09-13T17:43:56.017201+0000","last_became_active":"2024-09-13T17:43:36.971235+0000","last_became_peered":"2024-09-13T17:43:36.971235+0000","last_unstale":"2024-09-13T17:43:56.017201+0000","last_undegraded":"2024-09-13T17:43:56.017201+0000","last_fullsized":"2024-09-13T17:43:56.017201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:28.557515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524273+0000","last_change":"2024-09-13T17:43:56.036290+0000","last_active":"2024-09-13T17:43:57.524273+0000","last_peered":"2024-09-13T17:43:57.524273+0000","last_clean":"2024-09-13T17:43:57.524273+0000","last_became_active":"2024-09-13T17:43:56.036007+0000","last_became_peered":"2024-09-13T17:43:56.036007+0000","last_unstale":"2024-09-13T17:43:57.524273+0000","last_undegraded":"2024-09-13T17:43:57.524273+0000","last_fullsized":"2024-09-13T17:43:57.524273+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:18.417056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022462+0000","last_change":"2024-09-13T17:43:36.989729+0000","last_active":"2024-09-13T17:43:56.022462+0000","last_peered":"2024-09-13T17:43:56.022462+0000","last_clean":"2024-09-13T17:43:56.022462+0000","last_became_active":"2024-09-13T17:43:36.989287+0000","last_became_peered":"2024-09-13T17:43:36.989287+0000","last_unstale":"2024-09-13T17:43:56.022462+0000","last_undegraded":"2024-09-13T17:43:56.022462+0000","last_fullsized":"2024-09-13T17:43:56.022462+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023309+0000","last_change":"2024-09-13T17:43:56.050400+0000","last_active":"2024-09-13T17:43:57.023309+0000","last_peered":"2024-09-13T17:43:57.023309+0000","last_clean":"2024-09-13T17:43:57.023309+0000","last_became_active":"2024-09-13T17:43:56.048806+0000","last_became_peered":"2024-09-13T17:43:56.048806+0000","last_unstale":"2024-09-13T17:43:57.023309+0000","last_undegraded":"2024-09-13T17:43:57.023309+0000","last_fullsized":"2024-09-13T17:43:57.023309+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.183956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013483+0000","last_change":"2024-09-13T17:43:36.972925+0000","last_active":"2024-09-13T17:43:56.013483+0000","last_peered":"2024-09-13T17:43:56.013483+0000","last_clean":"2024-09-13T17:43:56.013483+0000","last_became_active":"2024-09-13T17:43:36.972388+0000","last_became_peered":"2024-09-13T17:43:36.972388+0000","last_unstale":"2024-09-13T17:43:56.013483+0000","last_undegraded":"2024-09-13T17:43:56.013483+0000","last_fullsized":"2024-09-13T17:43:56.013483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:30.822850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020051+0000","last_change":"2024-09-13T17:43:56.039613+0000","last_active":"2024-09-13T17:43:57.020051+0000","last_peered":"2024-09-13T17:43:57.020051+0000","last_clean":"2024-09-13T17:43:57.020051+0000","last_became_active":"2024-09-13T17:43:56.039255+0000","last_became_peered":"2024-09-13T17:43:56.039255+0000","last_unstale":"2024-09-13T17:43:57.020051+0000","last_undegraded":"2024-09-13T17:43:57.020051+0000","last_fullsized":"2024-09-13T17:43:57.020051+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:33.996348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014127+0000","last_change":"2024-09-13T17:43:36.995318+0000","last_active":"2024-09-13T17:43:56.014127+0000","last_peered":"2024-09-13T17:43:56.014127+0000","last_clean":"2024-09-13T17:43:56.014127+0000","last_became_active":"2024-09-13T17:43:36.995041+0000","last_became_peered":"2024-09-13T17:43:36.995041+0000","last_unstale":"2024-09-13T17:43:56.014127+0000","last_undegraded":"2024-09-13T17:43:56.014127+0000","last_fullsized":"2024-09-13T17:43:56.014127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:59.884186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523698+0000","last_change":"2024-09-13T17:43:56.034190+0000","last_active":"2024-09-13T17:43:57.523698+0000","last_peered":"2024-09-13T17:43:57.523698+0000","last_clean":"2024-09-13T17:43:57.523698+0000","last_became_active":"2024-09-13T17:43:56.031183+0000","last_became_peered":"2024-09-13T17:43:56.031183+0000","last_unstale":"2024-09-13T17:43:57.523698+0000","last_undegraded":"2024-09-13T17:43:57.523698+0000","last_fullsized":"2024-09-13T17:43:57.523698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:07:08.067994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022710+0000","last_change":"2024-09-13T17:43:36.981632+0000","last_active":"2024-09-13T17:43:56.022710+0000","last_peered":"2024-09-13T17:43:56.022710+0000","last_clean":"2024-09-13T17:43:56.022710+0000","last_became_active":"2024-09-13T17:43:36.977775+0000","last_became_peered":"2024-09-13T17:43:36.977775+0000","last_unstale":"2024-09-13T17:43:56.022710+0000","last_undegraded":"2024-09-13T17:43:56.022710+0000","last_fullsized":"2024-09-13T17:43:56.022710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.056493+0000","last_change":"2024-09-13T17:43:56.055026+0000","last_active":"2024-09-13T17:43:56.056493+0000","last_peered":"2024-09-13T17:43:56.056493+0000","last_clean":"2024-09-13T17:43:56.056493+0000","last_became_active":"2024-09-13T17:43:56.054419+0000","last_became_peered":"2024-09-13T17:43:56.054419+0000","last_unstale":"2024-09-13T17:43:56.056493+0000","last_undegraded":"2024-09-13T17:43:56.056493+0000","last_fullsized":"2024-09-13T17:43:56.056493+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:04.391077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024362+0000","last_change":"2024-09-13T17:43:36.976924+0000","last_active":"2024-09-13T17:43:56.024362+0000","last_peered":"2024-09-13T17:43:56.024362+0000","last_clean":"2024-09-13T17:43:56.024362+0000","last_became_active":"2024-09-13T17:43:36.975597+0000","last_became_peered":"2024-09-13T17:43:36.975597+0000","last_unstale":"2024-09-13T17:43:56.024362+0000","last_undegraded":"2024-09-13T17:43:56.024362+0000","last_fullsized":"2024-09-13T17:43:56.024362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:43.031842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019796+0000","last_change":"2024-09-13T17:43:56.046366+0000","last_active":"2024-09-13T17:43:57.019796+0000","last_peered":"2024-09-13T17:43:57.019796+0000","last_clean":"2024-09-13T17:43:57.019796+0000","last_became_active":"2024-09-13T17:43:56.044770+0000","last_became_peered":"2024-09-13T17:43:56.044770+0000","last_unstale":"2024-09-13T17:43:57.019796+0000","last_undegraded":"2024-09-13T17:43:57.019796+0000","last_fullsized":"2024-09-13T17:43:57.019796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:02.407638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015193+0000","last_change":"2024-09-13T17:43:36.996728+0000","last_active":"2024-09-13T17:43:56.015193+0000","last_peered":"2024-09-13T17:43:56.015193+0000","last_clean":"2024-09-13T17:43:56.015193+0000","last_became_active":"2024-09-13T17:43:36.996468+0000","last_became_peered":"2024-09-13T17:43:36.996468+0000","last_unstale":"2024-09-13T17:43:56.015193+0000","last_undegraded":"2024-09-13T17:43:56.015193+0000","last_fullsized":"2024-09-13T17:43:56.015193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:56.456764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522395+0000","last_change":"2024-09-13T17:43:56.040993+0000","last_active":"2024-09-13T17:43:57.522395+0000","last_peered":"2024-09-13T17:43:57.522395+0000","last_clean":"2024-09-13T17:43:57.522395+0000","last_became_active":"2024-09-13T17:43:56.037505+0000","last_became_peered":"2024-09-13T17:43:56.037505+0000","last_unstale":"2024-09-13T17:43:57.522395+0000","last_undegraded":"2024-09-13T17:43:57.522395+0000","last_fullsized":"2024-09-13T17:43:57.522395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:26.680526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652679+0000","last_change":"2024-09-13T17:43:36.991727+0000","last_active":"2024-09-13T17:43:55.652679+0000","last_peered":"2024-09-13T17:43:55.652679+0000","last_clean":"2024-09-13T17:43:55.652679+0000","last_became_active":"2024-09-13T17:43:36.988498+0000","last_became_peered":"2024-09-13T17:43:36.988498+0000","last_unstale":"2024-09-13T17:43:55.652679+0000","last_undegraded":"2024-09-13T17:43:55.652679+0000","last_fullsized":"2024-09-13T17:43:55.652679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018961+0000","last_change":"2024-09-13T17:43:56.044757+0000","last_active":"2024-09-13T17:43:57.018961+0000","last_peered":"2024-09-13T17:43:57.018961+0000","last_clean":"2024-09-13T17:43:57.018961+0000","last_became_active":"2024-09-13T17:43:56.044378+0000","last_became_peered":"2024-09-13T17:43:56.044378+0000","last_unstale":"2024-09-13T17:43:57.018961+0000","last_undegraded":"2024-09-13T17:43:57.018961+0000","last_fullsized":"2024-09-13T17:43:57.018961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.432317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.029101+0000","last_change":"2024-09-13T17:43:36.976795+0000","last_active":"2024-09-13T17:43:56.029101+0000","last_peered":"2024-09-13T17:43:56.029101+0000","last_clean":"2024-09-13T17:43:56.029101+0000","last_became_active":"2024-09-13T17:43:36.974036+0000","last_became_peered":"2024-09-13T17:43:36.974036+0000","last_unstale":"2024-09-13T17:43:56.029101+0000","last_undegraded":"2024-09-13T17:43:56.029101+0000","last_fullsized":"2024-09-13T17:43:56.029101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020905+0000","last_change":"2024-09-13T17:43:56.046721+0000","last_active":"2024-09-13T17:43:57.020905+0000","last_peered":"2024-09-13T17:43:57.020905+0000","last_clean":"2024-09-13T17:43:57.020905+0000","last_became_active":"2024-09-13T17:43:56.045178+0000","last_became_peered":"2024-09-13T17:43:56.045178+0000","last_unstale":"2024-09-13T17:43:57.020905+0000","last_undegraded":"2024-09-13T17:43:57.020905+0000","last_fullsized":"2024-09-13T17:43:57.020905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:17.824302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017410+0000","last_change":"2024-09-13T17:43:36.997635+0000","last_active":"2024-09-13T17:43:56.017410+0000","last_peered":"2024-09-13T17:43:56.017410+0000","last_clean":"2024-09-13T17:43:56.017410+0000","last_became_active":"2024-09-13T17:43:36.997014+0000","last_became_peered":"2024-09-13T17:43:36.997014+0000","last_unstale":"2024-09-13T17:43:56.017410+0000","last_undegraded":"2024-09-13T17:43:56.017410+0000","last_fullsized":"2024-09-13T17:43:56.017410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:36.971585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022460+0000","last_change":"2024-09-13T17:43:56.045478+0000","last_active":"2024-09-13T17:43:57.022460+0000","last_peered":"2024-09-13T17:43:57.022460+0000","last_clean":"2024-09-13T17:43:57.022460+0000","last_became_active":"2024-09-13T17:43:56.044526+0000","last_became_peered":"2024-09-13T17:43:56.044526+0000","last_unstale":"2024-09-13T17:43:57.022460+0000","last_undegraded":"2024-09-13T17:43:57.022460+0000","last_fullsized":"2024-09-13T17:43:57.022460+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:38.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023632+0000","last_change":"2024-09-13T17:43:36.989971+0000","last_active":"2024-09-13T17:43:56.023632+0000","last_peered":"2024-09-13T17:43:56.023632+0000","last_clean":"2024-09-13T17:43:56.023632+0000","last_became_active":"2024-09-13T17:43:36.989598+0000","last_became_peered":"2024-09-13T17:43:36.989598+0000","last_unstale":"2024-09-13T17:43:56.023632+0000","last_undegraded":"2024-09-13T17:43:56.023632+0000","last_fullsized":"2024-09-13T17:43:56.023632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527194+0000","last_change":"2024-09-13T17:43:56.036448+0000","last_active":"2024-09-13T17:43:57.527194+0000","last_peered":"2024-09-13T17:43:57.527194+0000","last_clean":"2024-09-13T17:43:57.527194+0000","last_became_active":"2024-09-13T17:43:56.035769+0000","last_became_peered":"2024-09-13T17:43:56.035769+0000","last_unstale":"2024-09-13T17:43:57.527194+0000","last_undegraded":"2024-09-13T17:43:57.527194+0000","last_fullsized":"2024-09-13T17:43:57.527194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:07.252720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014211+0000","last_change":"2024-09-13T17:43:36.998495+0000","last_active":"2024-09-13T17:43:56.014211+0000","last_peered":"2024-09-13T17:43:56.014211+0000","last_clean":"2024-09-13T17:43:56.014211+0000","last_became_active":"2024-09-13T17:43:36.998203+0000","last_became_peered":"2024-09-13T17:43:36.998203+0000","last_unstale":"2024-09-13T17:43:56.014211+0000","last_undegraded":"2024-09-13T17:43:56.014211+0000","last_fullsized":"2024-09-13T17:43:56.014211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:39.769029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020314+0000","last_change":"2024-09-13T17:43:56.046084+0000","last_active":"2024-09-13T17:43:57.020314+0000","last_peered":"2024-09-13T17:43:57.020314+0000","last_clean":"2024-09-13T17:43:57.020314+0000","last_became_active":"2024-09-13T17:43:56.044328+0000","last_became_peered":"2024-09-13T17:43:56.044328+0000","last_unstale":"2024-09-13T17:43:57.020314+0000","last_undegraded":"2024-09-13T17:43:57.020314+0000","last_fullsized":"2024-09-13T17:43:57.020314+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:16.263521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023142+0000","last_change":"2024-09-13T17:43:36.989784+0000","last_active":"2024-09-13T17:43:56.023142+0000","last_peered":"2024-09-13T17:43:56.023142+0000","last_clean":"2024-09-13T17:43:56.023142+0000","last_became_active":"2024-09-13T17:43:36.989513+0000","last_became_peered":"2024-09-13T17:43:36.989513+0000","last_unstale":"2024-09-13T17:43:56.023142+0000","last_undegraded":"2024-09-13T17:43:56.023142+0000","last_fullsized":"2024-09-13T17:43:56.023142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521734+0000","last_change":"2024-09-13T17:43:56.038766+0000","last_active":"2024-09-13T17:43:57.521734+0000","last_peered":"2024-09-13T17:43:57.521734+0000","last_clean":"2024-09-13T17:43:57.521734+0000","last_became_active":"2024-09-13T17:43:56.038508+0000","last_became_peered":"2024-09-13T17:43:56.038508+0000","last_unstale":"2024-09-13T17:43:57.521734+0000","last_undegraded":"2024-09-13T17:43:57.521734+0000","last_fullsized":"2024-09-13T17:43:57.521734+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:42.670857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024443+0000","last_change":"2024-09-13T17:43:36.970077+0000","last_active":"2024-09-13T17:43:56.024443+0000","last_peered":"2024-09-13T17:43:56.024443+0000","last_clean":"2024-09-13T17:43:56.024443+0000","last_became_active":"2024-09-13T17:43:36.969731+0000","last_became_peered":"2024-09-13T17:43:36.969731+0000","last_unstale":"2024-09-13T17:43:56.024443+0000","last_undegraded":"2024-09-13T17:43:56.024443+0000","last_fullsized":"2024-09-13T17:43:56.024443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020251+0000","last_change":"2024-09-13T17:43:56.048654+0000","last_active":"2024-09-13T17:43:57.020251+0000","last_peered":"2024-09-13T17:43:57.020251+0000","last_clean":"2024-09-13T17:43:57.020251+0000","last_became_active":"2024-09-13T17:43:56.048293+0000","last_became_peered":"2024-09-13T17:43:56.048293+0000","last_unstale":"2024-09-13T17:43:57.020251+0000","last_undegraded":"2024-09-13T17:43:57.020251+0000","last_fullsized":"2024-09-13T17:43:57.020251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.090860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015522+0000","last_change":"2024-09-13T17:43:36.999259+0000","last_active":"2024-09-13T17:43:56.015522+0000","last_peered":"2024-09-13T17:43:56.015522+0000","last_clean":"2024-09-13T17:43:56.015522+0000","last_became_active":"2024-09-13T17:43:36.998970+0000","last_became_peered":"2024-09-13T17:43:36.998970+0000","last_unstale":"2024-09-13T17:43:56.015522+0000","last_undegraded":"2024-09-13T17:43:56.015522+0000","last_fullsized":"2024-09-13T17:43:56.015522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:26.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018373+0000","last_change":"2024-09-13T17:43:56.044870+0000","last_active":"2024-09-13T17:43:57.018373+0000","last_peered":"2024-09-13T17:43:57.018373+0000","last_clean":"2024-09-13T17:43:57.018373+0000","last_became_active":"2024-09-13T17:43:56.044589+0000","last_became_peered":"2024-09-13T17:43:56.044589+0000","last_unstale":"2024-09-13T17:43:57.018373+0000","last_undegraded":"2024-09-13T17:43:57.018373+0000","last_fullsized":"2024-09-13T17:43:57.018373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:16:27.301097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015422+0000","last_change":"2024-09-13T17:43:36.972143+0000","last_active":"2024-09-13T17:43:56.015422+0000","last_peered":"2024-09-13T17:43:56.015422+0000","last_clean":"2024-09-13T17:43:56.015422+0000","last_became_active":"2024-09-13T17:43:36.971262+0000","last_became_peered":"2024-09-13T17:43:36.971262+0000","last_unstale":"2024-09-13T17:43:56.015422+0000","last_undegraded":"2024-09-13T17:43:56.015422+0000","last_fullsized":"2024-09-13T17:43:56.015422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:24.359313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521001+0000","last_change":"2024-09-13T17:43:56.031502+0000","last_active":"2024-09-13T17:43:57.521001+0000","last_peered":"2024-09-13T17:43:57.521001+0000","last_clean":"2024-09-13T17:43:57.521001+0000","last_became_active":"2024-09-13T17:43:56.029129+0000","last_became_peered":"2024-09-13T17:43:56.029129+0000","last_unstale":"2024-09-13T17:43:57.521001+0000","last_undegraded":"2024-09-13T17:43:57.521001+0000","last_fullsized":"2024-09-13T17:43:57.521001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:26.414401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017282+0000","last_change":"2024-09-13T17:43:36.998418+0000","last_active":"2024-09-13T17:43:56.017282+0000","last_peered":"2024-09-13T17:43:56.017282+0000","last_clean":"2024-09-13T17:43:56.017282+0000","last_became_active":"2024-09-13T17:43:36.998145+0000","last_became_peered":"2024-09-13T17:43:36.998145+0000","last_unstale":"2024-09-13T17:43:56.017282+0000","last_undegraded":"2024-09-13T17:43:56.017282+0000","last_fullsized":"2024-09-13T17:43:56.017282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:12.103252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.046015+0000","last_change":"2024-09-13T17:43:56.042674+0000","last_active":"2024-09-13T17:43:56.046015+0000","last_peered":"2024-09-13T17:43:56.046015+0000","last_clean":"2024-09-13T17:43:56.046015+0000","last_became_active":"2024-09-13T17:43:56.042401+0000","last_became_peered":"2024-09-13T17:43:56.042401+0000","last_unstale":"2024-09-13T17:43:56.046015+0000","last_undegraded":"2024-09-13T17:43:56.046015+0000","last_fullsized":"2024-09-13T17:43:56.046015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:29.676452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017585+0000","last_change":"2024-09-13T17:43:36.997792+0000","last_active":"2024-09-13T17:43:56.017585+0000","last_peered":"2024-09-13T17:43:56.017585+0000","last_clean":"2024-09-13T17:43:56.017585+0000","last_became_active":"2024-09-13T17:43:36.997361+0000","last_became_peered":"2024-09-13T17:43:36.997361+0000","last_unstale":"2024-09-13T17:43:56.017585+0000","last_undegraded":"2024-09-13T17:43:56.017585+0000","last_fullsized":"2024-09-13T17:43:56.017585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:40.506501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041600+0000","last_change":"2024-09-13T17:43:56.038807+0000","last_active":"2024-09-13T17:43:56.041600+0000","last_peered":"2024-09-13T17:43:56.041600+0000","last_clean":"2024-09-13T17:43:56.041600+0000","last_became_active":"2024-09-13T17:43:56.038214+0000","last_became_peered":"2024-09-13T17:43:56.038214+0000","last_unstale":"2024-09-13T17:43:56.041600+0000","last_undegraded":"2024-09-13T17:43:56.041600+0000","last_fullsized":"2024-09-13T17:43:56.041600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:07.599159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013167+0000","last_change":"2024-09-13T17:43:36.999611+0000","last_active":"2024-09-13T17:43:56.013167+0000","last_peered":"2024-09-13T17:43:56.013167+0000","last_clean":"2024-09-13T17:43:56.013167+0000","last_became_active":"2024-09-13T17:43:36.998996+0000","last_became_peered":"2024-09-13T17:43:36.998996+0000","last_unstale":"2024-09-13T17:43:56.013167+0000","last_undegraded":"2024-09-13T17:43:56.013167+0000","last_fullsized":"2024-09-13T17:43:56.013167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:40.880802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045463+0000","last_change":"2024-09-13T17:43:56.040629+0000","last_active":"2024-09-13T17:43:56.045463+0000","last_peered":"2024-09-13T17:43:56.045463+0000","last_clean":"2024-09-13T17:43:56.045463+0000","last_became_active":"2024-09-13T17:43:56.040236+0000","last_became_peered":"2024-09-13T17:43:56.040236+0000","last_unstale":"2024-09-13T17:43:56.045463+0000","last_undegraded":"2024-09-13T17:43:56.045463+0000","last_fullsized":"2024-09-13T17:43:56.045463+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:39.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014344+0000","last_change":"2024-09-13T17:43:36.996807+0000","last_active":"2024-09-13T17:43:56.014344+0000","last_peered":"2024-09-13T17:43:56.014344+0000","last_clean":"2024-09-13T17:43:56.014344+0000","last_became_active":"2024-09-13T17:43:36.996515+0000","last_became_peered":"2024-09-13T17:43:36.996515+0000","last_unstale":"2024-09-13T17:43:56.014344+0000","last_undegraded":"2024-09-13T17:43:56.014344+0000","last_fullsized":"2024-09-13T17:43:56.014344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:46.786049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.035419+0000","last_change":"2024-09-13T17:43:56.033234+0000","last_active":"2024-09-13T17:43:56.035419+0000","last_peered":"2024-09-13T17:43:56.035419+0000","last_clean":"2024-09-13T17:43:56.035419+0000","last_became_active":"2024-09-13T17:43:56.032969+0000","last_became_peered":"2024-09-13T17:43:56.032969+0000","last_unstale":"2024-09-13T17:43:56.035419+0000","last_undegraded":"2024-09-13T17:43:56.035419+0000","last_fullsized":"2024-09-13T17:43:56.035419+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.761232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.020435+0000","last_change":"2024-09-13T17:43:36.998726+0000","last_active":"2024-09-13T17:43:56.020435+0000","last_peered":"2024-09-13T17:43:56.020435+0000","last_clean":"2024-09-13T17:43:56.020435+0000","last_became_active":"2024-09-13T17:43:36.998283+0000","last_became_peered":"2024-09-13T17:43:36.998283+0000","last_unstale":"2024-09-13T17:43:56.020435+0000","last_undegraded":"2024-09-13T17:43:56.020435+0000","last_fullsized":"2024-09-13T17:43:56.020435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:03.727034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019395+0000","last_change":"2024-09-13T17:43:56.056236+0000","last_active":"2024-09-13T17:43:57.019395+0000","last_peered":"2024-09-13T17:43:57.019395+0000","last_clean":"2024-09-13T17:43:57.019395+0000","last_became_active":"2024-09-13T17:43:56.055988+0000","last_became_peered":"2024-09-13T17:43:56.055988+0000","last_unstale":"2024-09-13T17:43:57.019395+0000","last_undegraded":"2024-09-13T17:43:57.019395+0000","last_fullsized":"2024-09-13T17:43:57.019395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:57.966224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013619+0000","last_change":"2024-09-13T17:43:36.972979+0000","last_active":"2024-09-13T17:43:56.013619+0000","last_peered":"2024-09-13T17:43:56.013619+0000","last_clean":"2024-09-13T17:43:56.013619+0000","last_became_active":"2024-09-13T17:43:36.972625+0000","last_became_peered":"2024-09-13T17:43:36.972625+0000","last_unstale":"2024-09-13T17:43:56.013619+0000","last_undegraded":"2024-09-13T17:43:56.013619+0000","last_fullsized":"2024-09-13T17:43:56.013619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:52.901570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523282+0000","last_change":"2024-09-13T17:43:56.052186+0000","last_active":"2024-09-13T17:43:57.523282+0000","last_peered":"2024-09-13T17:43:57.523282+0000","last_clean":"2024-09-13T17:43:57.523282+0000","last_became_active":"2024-09-13T17:43:56.051554+0000","last_became_peered":"2024-09-13T17:43:56.051554+0000","last_unstale":"2024-09-13T17:43:57.523282+0000","last_undegraded":"2024-09-13T17:43:57.523282+0000","last_fullsized":"2024-09-13T17:43:57.523282+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:51.786165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653359+0000","last_change":"2024-09-13T17:43:36.988922+0000","last_active":"2024-09-13T17:43:55.653359+0000","last_peered":"2024-09-13T17:43:55.653359+0000","last_clean":"2024-09-13T17:43:55.653359+0000","last_became_active":"2024-09-13T17:43:36.988648+0000","last_became_peered":"2024-09-13T17:43:36.988648+0000","last_unstale":"2024-09-13T17:43:55.653359+0000","last_undegraded":"2024-09-13T17:43:55.653359+0000","last_fullsized":"2024-09-13T17:43:55.653359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.520682+0000","last_change":"2024-09-13T17:43:56.041845+0000","last_active":"2024-09-13T17:43:57.520682+0000","last_peered":"2024-09-13T17:43:57.520682+0000","last_clean":"2024-09-13T17:43:57.520682+0000","last_became_active":"2024-09-13T17:43:56.035755+0000","last_became_peered":"2024-09-13T17:43:56.035755+0000","last_unstale":"2024-09-13T17:43:57.520682+0000","last_undegraded":"2024-09-13T17:43:57.520682+0000","last_fullsized":"2024-09-13T17:43:57.520682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:58.808502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026496+0000","last_change":"2024-09-13T17:43:36.974086+0000","last_active":"2024-09-13T17:43:56.026496+0000","last_peered":"2024-09-13T17:43:56.026496+0000","last_clean":"2024-09-13T17:43:56.026496+0000","last_became_active":"2024-09-13T17:43:36.973536+0000","last_became_peered":"2024-09-13T17:43:36.973536+0000","last_unstale":"2024-09-13T17:43:56.026496+0000","last_undegraded":"2024-09-13T17:43:56.026496+0000","last_fullsized":"2024-09-13T17:43:56.026496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:22:24.478487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.520997+0000","last_change":"2024-09-13T17:43:56.051373+0000","last_active":"2024-09-13T17:43:57.520997+0000","last_peered":"2024-09-13T17:43:57.520997+0000","last_clean":"2024-09-13T17:43:57.520997+0000","last_became_active":"2024-09-13T17:43:56.045228+0000","last_became_peered":"2024-09-13T17:43:56.045228+0000","last_unstale":"2024-09-13T17:43:57.520997+0000","last_undegraded":"2024-09-13T17:43:57.520997+0000","last_fullsized":"2024-09-13T17:43:57.520997+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:09.551521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017363+0000","last_change":"2024-09-13T17:43:36.972254+0000","last_active":"2024-09-13T17:43:56.017363+0000","last_peered":"2024-09-13T17:43:56.017363+0000","last_clean":"2024-09-13T17:43:56.017363+0000","last_became_active":"2024-09-13T17:43:36.971289+0000","last_became_peered":"2024-09-13T17:43:36.971289+0000","last_unstale":"2024-09-13T17:43:56.017363+0000","last_undegraded":"2024-09-13T17:43:56.017363+0000","last_fullsized":"2024-09-13T17:43:56.017363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:09.542526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4994,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":197,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738395,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":155325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.498}]},{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738396,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":995328,"data_stored":734655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.443},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.46200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738395,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":1011712,"data_stored":740859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29899999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:00.000 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:00.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.144+0000 7f6879f84640 1 Processor -- start 2024-09-13T17:44:00.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 -- start start 2024-09-13T17:44:00.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f687406f6d0 0x7f6874134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687406e130 con 0x7f687409dc20 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68740a6750 con 0x7f687406f6d0 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f687406f6d0 0x7f6874134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f687406f6d0 0x7f6874134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49012/0 (socket says 172.21.15.96:49012) 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 -- 172.21.15.96:0/1227261157 learned_addr learned my addr 172.21.15.96:0/1227261157 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f686affd640 1 --1- 172.21.15.96:0/1227261157 >> v1:172.21.15.96:6789/0 conn(0x7f687409dc20 0x7f6874134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57722/0 (socket says 172.21.15.96:57722) 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 -- 172.21.15.96:0/1227261157 >> v1:172.21.15.96:6789/0 conn(0x7f687409dc20 legacy=0x7f6874134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 -- 172.21.15.96:0/1227261157 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68740712e0 con 0x7f687406f6d0 2024-09-13T17:44:00.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f68737fe640 1 --2- 172.21.15.96:0/1227261157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f6874134ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f685c0097f0 tx=0x7f685c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ea6325f27d231ab server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6872ffd640 1 -- 172.21.15.96:0/1227261157 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0ab400 con 0x7f687406f6d0 2024-09-13T17:44:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6872ffd640 1 -- 172.21.15.96:0/1227261157 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c0a7050 con 0x7f687406f6d0 2024-09-13T17:44:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6872ffd640 1 -- 172.21.15.96:0/1227261157 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0ab6b0 con 0x7f687406f6d0 2024-09-13T17:44:00.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 -- 172.21.15.96:0/1227261157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 msgr2=0x7f6874134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.148+0000 7f6879f84640 1 --2- 172.21.15.96:0/1227261157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f6874134ab0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f685c0097f0 tx=0x7f685c01c450 comp rx=0 tx=0).stop 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/1227261157 shutdown_connections 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 --2- 172.21.15.96:0/1227261157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f6874134ab0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/1227261157 >> 172.21.15.96:0/1227261157 conn(0x7f6874098a60 msgr2=0x7f687409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/1227261157 shutdown_connections 2024-09-13T17:44:00.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/1227261157 wait complete. 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 Processor -- start 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- start start 2024-09-13T17:44:00.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68741218d0 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f68737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f68737fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49028/0 (socket says 172.21.15.96:49028) 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f68737fe640 1 -- 172.21.15.96:0/803072452 learned_addr learned my addr 172.21.15.96:0/803072452 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f68737fe640 1 -- 172.21.15.96:0/803072452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685c0094d0 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f68737fe640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f685c01a080 tx=0x7f685c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0b6420 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685c0b6580 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6874139100 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f685c0b6840 con 0x7f687406f6d0 2024-09-13T17:44:00.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6874120f90 con 0x7f687406f6d0 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f685c0b2040 con 0x7f687406f6d0 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687411edb0 con 0x7f687406f6d0 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 0x7f6858037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f685c0e23c0 con 0x7f687406f6d0 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f686affd640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 0x7f6858037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.152+0000 7f686affd640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 0x7f6858037a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6864001600 tx=0x7f6864007a50 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.156+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f685c0b69a0 con 0x7f687406f6d0 2024-09-13T17:44:00.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.300+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 --> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6874135910 con 0x7f68580355c0 2024-09-13T17:44:00.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.312+0000 7f6870ff9640 1 -- 172.21.15.96:0/803072452 <== mgr.4821 v2:172.21.15.96:6824/1425974344 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+486222 (secure 0 0 0) 0x7f68640070d0 con 0x7f68580355c0 2024-09-13T17:44:00.320 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:00.323 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:44:00.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 msgr2=0x7f6858037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 0x7f6858037a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6864001600 tx=0x7f6864007a50 comp rx=0 tx=0).stop 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 msgr2=0x7f687411d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f685c01a080 tx=0x7f685c01ac90 comp rx=0 tx=0).stop 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 shutdown_connections 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68580355c0 0x7f6858037a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 --2- 172.21.15.96:0/803072452 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f687406f6d0 0x7f687411d370 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 >> 172.21.15.96:0/803072452 conn(0x7f6874098a60 msgr2=0x7f6874139810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 shutdown_connections 2024-09-13T17:44:00.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.316+0000 7f6879f84640 1 -- 172.21.15.96:0/803072452 wait complete. 2024-09-13T17:44:00.335 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-13T17:43:59.602456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597724,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":292,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":213,"ondisk_log_size":213,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":341,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":91664,"kb_used_data":2392,"kb_used_omap":4,"kb_used_meta":89211,"kb_avail":283023856,"statfs":{"total":289910292480,"available":289816428544,"internally_reserved":0,"allocated":2449408,"data_stored":1630839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":91352403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":65,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.982939"},"pg_stats":[{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041642+0000","last_change":"2024-09-13T17:43:56.039642+0000","last_active":"2024-09-13T17:43:56.041642+0000","last_peered":"2024-09-13T17:43:56.041642+0000","last_clean":"2024-09-13T17:43:56.041642+0000","last_became_active":"2024-09-13T17:43:56.039348+0000","last_became_peered":"2024-09-13T17:43:56.039348+0000","last_unstale":"2024-09-13T17:43:56.041642+0000","last_undegraded":"2024-09-13T17:43:56.041642+0000","last_fullsized":"2024-09-13T17:43:56.041642+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:06.416508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"36'4","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.028580+0000","last_change":"2024-09-13T17:43:36.597308+0000","last_active":"2024-09-13T17:43:56.028580+0000","last_peered":"2024-09-13T17:43:56.028580+0000","last_clean":"2024-09-13T17:43:56.028580+0000","last_became_active":"2024-09-13T17:43:36.596591+0000","last_became_peered":"2024-09-13T17:43:36.596591+0000","last_unstale":"2024-09-13T17:43:56.028580+0000","last_undegraded":"2024-09-13T17:43:56.028580+0000","last_fullsized":"2024-09-13T17:43:56.028580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:55:03.103037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652774+0000","last_change":"2024-09-13T17:43:36.997889+0000","last_active":"2024-09-13T17:43:55.652774+0000","last_peered":"2024-09-13T17:43:55.652774+0000","last_clean":"2024-09-13T17:43:55.652774+0000","last_became_active":"2024-09-13T17:43:36.997671+0000","last_became_peered":"2024-09-13T17:43:36.997671+0000","last_unstale":"2024-09-13T17:43:55.652774+0000","last_undegraded":"2024-09-13T17:43:55.652774+0000","last_fullsized":"2024-09-13T17:43:55.652774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018830+0000","last_change":"2024-09-13T17:43:55.672769+0000","last_active":"2024-09-13T17:43:57.018830+0000","last_peered":"2024-09-13T17:43:57.018830+0000","last_clean":"2024-09-13T17:43:57.018830+0000","last_became_active":"2024-09-13T17:43:55.672449+0000","last_became_peered":"2024-09-13T17:43:55.672449+0000","last_unstale":"2024-09-13T17:43:57.018830+0000","last_undegraded":"2024-09-13T17:43:57.018830+0000","last_fullsized":"2024-09-13T17:43:57.018830+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:58.723057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523341+0000","last_change":"2024-09-13T17:43:56.036340+0000","last_active":"2024-09-13T17:43:57.523341+0000","last_peered":"2024-09-13T17:43:57.523341+0000","last_clean":"2024-09-13T17:43:57.523341+0000","last_became_active":"2024-09-13T17:43:56.036034+0000","last_became_peered":"2024-09-13T17:43:56.036034+0000","last_unstale":"2024-09-13T17:43:57.523341+0000","last_undegraded":"2024-09-13T17:43:57.523341+0000","last_fullsized":"2024-09-13T17:43:57.523341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:38:43.427429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"36'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653024+0000","last_change":"2024-09-13T17:43:36.593051+0000","last_active":"2024-09-13T17:43:55.653024+0000","last_peered":"2024-09-13T17:43:55.653024+0000","last_clean":"2024-09-13T17:43:55.653024+0000","last_became_active":"2024-09-13T17:43:36.592210+0000","last_became_peered":"2024-09-13T17:43:36.592210+0000","last_unstale":"2024-09-13T17:43:55.653024+0000","last_undegraded":"2024-09-13T17:43:55.653024+0000","last_fullsized":"2024-09-13T17:43:55.653024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:00:07.519036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017125+0000","last_change":"2024-09-13T17:43:36.995868+0000","last_active":"2024-09-13T17:43:56.017125+0000","last_peered":"2024-09-13T17:43:56.017125+0000","last_clean":"2024-09-13T17:43:56.017125+0000","last_became_active":"2024-09-13T17:43:36.995304+0000","last_became_peered":"2024-09-13T17:43:36.995304+0000","last_unstale":"2024-09-13T17:43:56.017125+0000","last_undegraded":"2024-09-13T17:43:56.017125+0000","last_fullsized":"2024-09-13T17:43:56.017125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:40.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"41'8","reported_seq":19,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522247+0000","last_change":"2024-09-13T17:43:55.657425+0000","last_active":"2024-09-13T17:43:57.522247+0000","last_peered":"2024-09-13T17:43:57.522247+0000","last_clean":"2024-09-13T17:43:57.522247+0000","last_became_active":"2024-09-13T17:43:55.657150+0000","last_became_peered":"2024-09-13T17:43:55.657150+0000","last_unstale":"2024-09-13T17:43:57.522247+0000","last_undegraded":"2024-09-13T17:43:57.522247+0000","last_fullsized":"2024-09-13T17:43:57.522247+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:39:52.217667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.056514+0000","last_change":"2024-09-13T17:43:56.055208+0000","last_active":"2024-09-13T17:43:56.056514+0000","last_peered":"2024-09-13T17:43:56.056514+0000","last_clean":"2024-09-13T17:43:56.056514+0000","last_became_active":"2024-09-13T17:43:56.054775+0000","last_became_peered":"2024-09-13T17:43:56.054775+0000","last_unstale":"2024-09-13T17:43:56.056514+0000","last_undegraded":"2024-09-13T17:43:56.056514+0000","last_fullsized":"2024-09-13T17:43:56.056514+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:06.353192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652193+0000","last_change":"2024-09-13T17:43:36.607525+0000","last_active":"2024-09-13T17:43:55.652193+0000","last_peered":"2024-09-13T17:43:55.652193+0000","last_clean":"2024-09-13T17:43:55.652193+0000","last_became_active":"2024-09-13T17:43:36.606688+0000","last_became_peered":"2024-09-13T17:43:36.606688+0000","last_unstale":"2024-09-13T17:43:55.652193+0000","last_undegraded":"2024-09-13T17:43:55.652193+0000","last_fullsized":"2024-09-13T17:43:55.652193+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:12:34.433256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024202+0000","last_change":"2024-09-13T17:43:36.994621+0000","last_active":"2024-09-13T17:43:56.024202+0000","last_peered":"2024-09-13T17:43:56.024202+0000","last_clean":"2024-09-13T17:43:56.024202+0000","last_became_active":"2024-09-13T17:43:36.993891+0000","last_became_peered":"2024-09-13T17:43:36.993891+0000","last_unstale":"2024-09-13T17:43:56.024202+0000","last_undegraded":"2024-09-13T17:43:56.024202+0000","last_fullsized":"2024-09-13T17:43:56.024202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"41'2","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.049310+0000","last_change":"2024-09-13T17:43:55.673793+0000","last_active":"2024-09-13T17:43:56.049310+0000","last_peered":"2024-09-13T17:43:56.049310+0000","last_clean":"2024-09-13T17:43:56.049310+0000","last_became_active":"2024-09-13T17:43:55.673508+0000","last_became_peered":"2024-09-13T17:43:55.673508+0000","last_unstale":"2024-09-13T17:43:56.049310+0000","last_undegraded":"2024-09-13T17:43:56.049310+0000","last_fullsized":"2024-09-13T17:43:56.049310+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:21.777386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019788+0000","last_change":"2024-09-13T17:43:56.046175+0000","last_active":"2024-09-13T17:43:57.019788+0000","last_peered":"2024-09-13T17:43:57.019788+0000","last_clean":"2024-09-13T17:43:57.019788+0000","last_became_active":"2024-09-13T17:43:56.044393+0000","last_became_peered":"2024-09-13T17:43:56.044393+0000","last_unstale":"2024-09-13T17:43:57.019788+0000","last_undegraded":"2024-09-13T17:43:57.019788+0000","last_fullsized":"2024-09-13T17:43:57.019788+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:29.059066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652772+0000","last_change":"2024-09-13T17:43:36.604117+0000","last_active":"2024-09-13T17:43:55.652772+0000","last_peered":"2024-09-13T17:43:55.652772+0000","last_clean":"2024-09-13T17:43:55.652772+0000","last_became_active":"2024-09-13T17:43:36.603754+0000","last_became_peered":"2024-09-13T17:43:36.603754+0000","last_unstale":"2024-09-13T17:43:55.652772+0000","last_undegraded":"2024-09-13T17:43:55.652772+0000","last_fullsized":"2024-09-13T17:43:55.652772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:48.898545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653156+0000","last_change":"2024-09-13T17:43:36.987727+0000","last_active":"2024-09-13T17:43:55.653156+0000","last_peered":"2024-09-13T17:43:55.653156+0000","last_clean":"2024-09-13T17:43:55.653156+0000","last_became_active":"2024-09-13T17:43:36.987360+0000","last_became_peered":"2024-09-13T17:43:36.987360+0000","last_unstale":"2024-09-13T17:43:55.653156+0000","last_undegraded":"2024-09-13T17:43:55.653156+0000","last_fullsized":"2024-09-13T17:43:55.653156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019087+0000","last_change":"2024-09-13T17:43:55.671195+0000","last_active":"2024-09-13T17:43:57.019087+0000","last_peered":"2024-09-13T17:43:57.019087+0000","last_clean":"2024-09-13T17:43:57.019087+0000","last_became_active":"2024-09-13T17:43:55.670558+0000","last_became_peered":"2024-09-13T17:43:55.670558+0000","last_unstale":"2024-09-13T17:43:57.019087+0000","last_undegraded":"2024-09-13T17:43:57.019087+0000","last_fullsized":"2024-09-13T17:43:57.019087+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:21.091910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527385+0000","last_change":"2024-09-13T17:43:56.040097+0000","last_active":"2024-09-13T17:43:57.527385+0000","last_peered":"2024-09-13T17:43:57.527385+0000","last_clean":"2024-09-13T17:43:57.527385+0000","last_became_active":"2024-09-13T17:43:56.039583+0000","last_became_peered":"2024-09-13T17:43:56.039583+0000","last_unstale":"2024-09-13T17:43:57.527385+0000","last_undegraded":"2024-09-13T17:43:57.527385+0000","last_fullsized":"2024-09-13T17:43:57.527385+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:02:17.654815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651196+0000","last_change":"2024-09-13T17:43:36.991670+0000","last_active":"2024-09-13T17:43:55.651196+0000","last_peered":"2024-09-13T17:43:55.651196+0000","last_clean":"2024-09-13T17:43:55.651196+0000","last_became_active":"2024-09-13T17:43:36.987763+0000","last_became_peered":"2024-09-13T17:43:36.987763+0000","last_unstale":"2024-09-13T17:43:55.651196+0000","last_undegraded":"2024-09-13T17:43:55.651196+0000","last_fullsized":"2024-09-13T17:43:55.651196+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013749+0000","last_change":"2024-09-13T17:43:36.612138+0000","last_active":"2024-09-13T17:43:56.013749+0000","last_peered":"2024-09-13T17:43:56.013749+0000","last_clean":"2024-09-13T17:43:56.013749+0000","last_became_active":"2024-09-13T17:43:36.609873+0000","last_became_peered":"2024-09-13T17:43:36.609873+0000","last_unstale":"2024-09-13T17:43:56.013749+0000","last_undegraded":"2024-09-13T17:43:56.013749+0000","last_fullsized":"2024-09-13T17:43:56.013749+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:49.578767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022188+0000","last_change":"2024-09-13T17:43:55.670827+0000","last_active":"2024-09-13T17:43:57.022188+0000","last_peered":"2024-09-13T17:43:57.022188+0000","last_clean":"2024-09-13T17:43:57.022188+0000","last_became_active":"2024-09-13T17:43:55.670513+0000","last_became_peered":"2024-09-13T17:43:55.670513+0000","last_unstale":"2024-09-13T17:43:57.022188+0000","last_undegraded":"2024-09-13T17:43:57.022188+0000","last_fullsized":"2024-09-13T17:43:57.022188+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:58.556852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045442+0000","last_change":"2024-09-13T17:43:56.040555+0000","last_active":"2024-09-13T17:43:56.045442+0000","last_peered":"2024-09-13T17:43:56.045442+0000","last_clean":"2024-09-13T17:43:56.045442+0000","last_became_active":"2024-09-13T17:43:56.040212+0000","last_became_peered":"2024-09-13T17:43:56.040212+0000","last_unstale":"2024-09-13T17:43:56.045442+0000","last_undegraded":"2024-09-13T17:43:56.045442+0000","last_fullsized":"2024-09-13T17:43:56.045442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:29.059066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022782+0000","last_change":"2024-09-13T17:43:36.977463+0000","last_active":"2024-09-13T17:43:56.022782+0000","last_peered":"2024-09-13T17:43:56.022782+0000","last_clean":"2024-09-13T17:43:56.022782+0000","last_became_active":"2024-09-13T17:43:36.977207+0000","last_became_peered":"2024-09-13T17:43:36.977207+0000","last_unstale":"2024-09-13T17:43:56.022782+0000","last_undegraded":"2024-09-13T17:43:56.022782+0000","last_fullsized":"2024-09-13T17:43:56.022782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:02:51.176544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651545+0000","last_change":"2024-09-13T17:43:36.607315+0000","last_active":"2024-09-13T17:43:55.651545+0000","last_peered":"2024-09-13T17:43:55.651545+0000","last_clean":"2024-09-13T17:43:55.651545+0000","last_became_active":"2024-09-13T17:43:36.606824+0000","last_became_peered":"2024-09-13T17:43:36.606824+0000","last_unstale":"2024-09-13T17:43:55.651545+0000","last_undegraded":"2024-09-13T17:43:55.651545+0000","last_fullsized":"2024-09-13T17:43:55.651545+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:07:30.362407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024440+0000","last_change":"2024-09-13T17:43:55.674375+0000","last_active":"2024-09-13T17:43:56.024440+0000","last_peered":"2024-09-13T17:43:56.024440+0000","last_clean":"2024-09-13T17:43:56.024440+0000","last_became_active":"2024-09-13T17:43:55.673757+0000","last_became_peered":"2024-09-13T17:43:55.673757+0000","last_unstale":"2024-09-13T17:43:56.024440+0000","last_undegraded":"2024-09-13T17:43:56.024440+0000","last_fullsized":"2024-09-13T17:43:56.024440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:57.819380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522842+0000","last_change":"2024-09-13T17:43:56.045854+0000","last_active":"2024-09-13T17:43:57.522842+0000","last_peered":"2024-09-13T17:43:57.522842+0000","last_clean":"2024-09-13T17:43:57.522842+0000","last_became_active":"2024-09-13T17:43:56.041173+0000","last_became_peered":"2024-09-13T17:43:56.041173+0000","last_unstale":"2024-09-13T17:43:57.522842+0000","last_undegraded":"2024-09-13T17:43:57.522842+0000","last_fullsized":"2024-09-13T17:43:57.522842+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:00.766965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653358+0000","last_change":"2024-09-13T17:43:36.988413+0000","last_active":"2024-09-13T17:43:55.653358+0000","last_peered":"2024-09-13T17:43:55.653358+0000","last_clean":"2024-09-13T17:43:55.653358+0000","last_became_active":"2024-09-13T17:43:36.987084+0000","last_became_peered":"2024-09-13T17:43:36.987084+0000","last_unstale":"2024-09-13T17:43:55.653358+0000","last_undegraded":"2024-09-13T17:43:55.653358+0000","last_fullsized":"2024-09-13T17:43:55.653358+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"37'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653242+0000","last_change":"2024-09-13T17:43:36.596762+0000","last_active":"2024-09-13T17:43:55.653242+0000","last_peered":"2024-09-13T17:43:55.653242+0000","last_clean":"2024-09-13T17:43:55.653242+0000","last_became_active":"2024-09-13T17:43:36.594514+0000","last_became_peered":"2024-09-13T17:43:36.594514+0000","last_unstale":"2024-09-13T17:43:55.653242+0000","last_undegraded":"2024-09-13T17:43:55.653242+0000","last_fullsized":"2024-09-13T17:43:55.653242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:30:50.353395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017417+0000","last_change":"2024-09-13T17:43:55.666577+0000","last_active":"2024-09-13T17:43:56.017417+0000","last_peered":"2024-09-13T17:43:56.017417+0000","last_clean":"2024-09-13T17:43:56.017417+0000","last_became_active":"2024-09-13T17:43:55.666281+0000","last_became_peered":"2024-09-13T17:43:55.666281+0000","last_unstale":"2024-09-13T17:43:56.017417+0000","last_undegraded":"2024-09-13T17:43:56.017417+0000","last_fullsized":"2024-09-13T17:43:56.017417+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:01:58.723057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034241+0000","last_change":"2024-09-13T17:43:56.029459+0000","last_active":"2024-09-13T17:43:56.034241+0000","last_peered":"2024-09-13T17:43:56.034241+0000","last_clean":"2024-09-13T17:43:56.034241+0000","last_became_active":"2024-09-13T17:43:56.029291+0000","last_became_peered":"2024-09-13T17:43:56.029291+0000","last_unstale":"2024-09-13T17:43:56.034241+0000","last_undegraded":"2024-09-13T17:43:56.034241+0000","last_fullsized":"2024-09-13T17:43:56.034241+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:07.604989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015004+0000","last_change":"2024-09-13T17:43:36.601888+0000","last_active":"2024-09-13T17:43:56.015004+0000","last_peered":"2024-09-13T17:43:56.015004+0000","last_clean":"2024-09-13T17:43:56.015004+0000","last_became_active":"2024-09-13T17:43:36.601588+0000","last_became_peered":"2024-09-13T17:43:36.601588+0000","last_unstale":"2024-09-13T17:43:56.015004+0000","last_undegraded":"2024-09-13T17:43:56.015004+0000","last_fullsized":"2024-09-13T17:43:56.015004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:05.159220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022938+0000","last_change":"2024-09-13T17:43:36.976896+0000","last_active":"2024-09-13T17:43:56.022938+0000","last_peered":"2024-09-13T17:43:56.022938+0000","last_clean":"2024-09-13T17:43:56.022938+0000","last_became_active":"2024-09-13T17:43:36.974032+0000","last_became_peered":"2024-09-13T17:43:36.974032+0000","last_unstale":"2024-09-13T17:43:56.022938+0000","last_undegraded":"2024-09-13T17:43:56.022938+0000","last_fullsized":"2024-09-13T17:43:56.022938+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:49:52.403736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.15","version":"41'3","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020547+0000","last_change":"2024-09-13T17:43:55.674507+0000","last_active":"2024-09-13T17:43:57.020547+0000","last_peered":"2024-09-13T17:43:57.020547+0000","last_clean":"2024-09-13T17:43:57.020547+0000","last_became_active":"2024-09-13T17:43:55.674231+0000","last_became_peered":"2024-09-13T17:43:55.674231+0000","last_unstale":"2024-09-13T17:43:57.020547+0000","last_undegraded":"2024-09-13T17:43:57.020547+0000","last_fullsized":"2024-09-13T17:43:57.020547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:30.153604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.057152+0000","last_change":"2024-09-13T17:43:56.055279+0000","last_active":"2024-09-13T17:43:56.057152+0000","last_peered":"2024-09-13T17:43:56.057152+0000","last_clean":"2024-09-13T17:43:56.057152+0000","last_became_active":"2024-09-13T17:43:56.055033+0000","last_became_peered":"2024-09-13T17:43:56.055033+0000","last_unstale":"2024-09-13T17:43:56.057152+0000","last_undegraded":"2024-09-13T17:43:56.057152+0000","last_fullsized":"2024-09-13T17:43:56.057152+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:08.974623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014370+0000","last_change":"2024-09-13T17:43:36.999031+0000","last_active":"2024-09-13T17:43:56.014370+0000","last_peered":"2024-09-13T17:43:56.014370+0000","last_clean":"2024-09-13T17:43:56.014370+0000","last_became_active":"2024-09-13T17:43:36.998251+0000","last_became_peered":"2024-09-13T17:43:36.998251+0000","last_unstale":"2024-09-13T17:43:56.014370+0000","last_undegraded":"2024-09-13T17:43:56.014370+0000","last_fullsized":"2024-09-13T17:43:56.014370+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:32.921571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015110+0000","last_change":"2024-09-13T17:43:36.612328+0000","last_active":"2024-09-13T17:43:56.015110+0000","last_peered":"2024-09-13T17:43:56.015110+0000","last_clean":"2024-09-13T17:43:56.015110+0000","last_became_active":"2024-09-13T17:43:36.610325+0000","last_became_peered":"2024-09-13T17:43:36.610325+0000","last_unstale":"2024-09-13T17:43:56.015110+0000","last_undegraded":"2024-09-13T17:43:56.015110+0000","last_fullsized":"2024-09-13T17:43:56.015110+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:31.356170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523251+0000","last_change":"2024-09-13T17:43:55.663340+0000","last_active":"2024-09-13T17:43:57.523251+0000","last_peered":"2024-09-13T17:43:57.523251+0000","last_clean":"2024-09-13T17:43:57.523251+0000","last_became_active":"2024-09-13T17:43:55.660795+0000","last_became_peered":"2024-09-13T17:43:55.660795+0000","last_unstale":"2024-09-13T17:43:57.523251+0000","last_undegraded":"2024-09-13T17:43:57.523251+0000","last_fullsized":"2024-09-13T17:43:57.523251+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:46:23.130203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020905+0000","last_change":"2024-09-13T17:43:56.046809+0000","last_active":"2024-09-13T17:43:57.020905+0000","last_peered":"2024-09-13T17:43:57.020905+0000","last_clean":"2024-09-13T17:43:57.020905+0000","last_became_active":"2024-09-13T17:43:56.046375+0000","last_became_peered":"2024-09-13T17:43:56.046375+0000","last_unstale":"2024-09-13T17:43:57.020905+0000","last_undegraded":"2024-09-13T17:43:57.020905+0000","last_fullsized":"2024-09-13T17:43:57.020905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:13:08.974623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022281+0000","last_change":"2024-09-13T17:43:36.990056+0000","last_active":"2024-09-13T17:43:56.022281+0000","last_peered":"2024-09-13T17:43:56.022281+0000","last_clean":"2024-09-13T17:43:56.022281+0000","last_became_active":"2024-09-13T17:43:36.989640+0000","last_became_peered":"2024-09-13T17:43:36.989640+0000","last_unstale":"2024-09-13T17:43:56.022281+0000","last_undegraded":"2024-09-13T17:43:56.022281+0000","last_fullsized":"2024-09-13T17:43:56.022281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012938+0000","last_change":"2024-09-13T17:43:36.612085+0000","last_active":"2024-09-13T17:43:56.012938+0000","last_peered":"2024-09-13T17:43:56.012938+0000","last_clean":"2024-09-13T17:43:56.012938+0000","last_became_active":"2024-09-13T17:43:36.609135+0000","last_became_peered":"2024-09-13T17:43:36.609135+0000","last_unstale":"2024-09-13T17:43:56.012938+0000","last_undegraded":"2024-09-13T17:43:56.012938+0000","last_fullsized":"2024-09-13T17:43:56.012938+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:22.346224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"41'2","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.047022+0000","last_change":"2024-09-13T17:43:55.673288+0000","last_active":"2024-09-13T17:43:56.047022+0000","last_peered":"2024-09-13T17:43:56.047022+0000","last_clean":"2024-09-13T17:43:56.047022+0000","last_became_active":"2024-09-13T17:43:55.672919+0000","last_became_peered":"2024-09-13T17:43:55.672919+0000","last_unstale":"2024-09-13T17:43:56.047022+0000","last_undegraded":"2024-09-13T17:43:56.047022+0000","last_fullsized":"2024-09-13T17:43:56.047022+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:44.906793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524232+0000","last_change":"2024-09-13T17:43:56.053427+0000","last_active":"2024-09-13T17:43:57.524232+0000","last_peered":"2024-09-13T17:43:57.524232+0000","last_clean":"2024-09-13T17:43:57.524232+0000","last_became_active":"2024-09-13T17:43:56.051828+0000","last_became_peered":"2024-09-13T17:43:56.051828+0000","last_unstale":"2024-09-13T17:43:57.524232+0000","last_undegraded":"2024-09-13T17:43:57.524232+0000","last_fullsized":"2024-09-13T17:43:57.524232+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:34.985529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024628+0000","last_change":"2024-09-13T17:43:36.973857+0000","last_active":"2024-09-13T17:43:56.024628+0000","last_peered":"2024-09-13T17:43:56.024628+0000","last_clean":"2024-09-13T17:43:56.024628+0000","last_became_active":"2024-09-13T17:43:36.973595+0000","last_became_peered":"2024-09-13T17:43:36.973595+0000","last_unstale":"2024-09-13T17:43:56.024628+0000","last_undegraded":"2024-09-13T17:43:56.024628+0000","last_fullsized":"2024-09-13T17:43:56.024628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:06.165795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653699+0000","last_change":"2024-09-13T17:43:36.592618+0000","last_active":"2024-09-13T17:43:55.653699+0000","last_peered":"2024-09-13T17:43:55.653699+0000","last_clean":"2024-09-13T17:43:55.653699+0000","last_became_active":"2024-09-13T17:43:36.592303+0000","last_became_peered":"2024-09-13T17:43:36.592303+0000","last_unstale":"2024-09-13T17:43:55.653699+0000","last_undegraded":"2024-09-13T17:43:55.653699+0000","last_fullsized":"2024-09-13T17:43:55.653699+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:16.952605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015889+0000","last_change":"2024-09-13T17:43:55.673141+0000","last_active":"2024-09-13T17:43:56.015889+0000","last_peered":"2024-09-13T17:43:56.015889+0000","last_clean":"2024-09-13T17:43:56.015889+0000","last_became_active":"2024-09-13T17:43:55.672869+0000","last_became_peered":"2024-09-13T17:43:55.672869+0000","last_unstale":"2024-09-13T17:43:56.015889+0000","last_undegraded":"2024-09-13T17:43:56.015889+0000","last_fullsized":"2024-09-13T17:43:56.015889+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:14:17.934538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526967+0000","last_change":"2024-09-13T17:43:56.037283+0000","last_active":"2024-09-13T17:43:57.526967+0000","last_peered":"2024-09-13T17:43:57.526967+0000","last_clean":"2024-09-13T17:43:57.526967+0000","last_became_active":"2024-09-13T17:43:56.036625+0000","last_became_peered":"2024-09-13T17:43:56.036625+0000","last_unstale":"2024-09-13T17:43:57.526967+0000","last_undegraded":"2024-09-13T17:43:57.526967+0000","last_fullsized":"2024-09-13T17:43:57.526967+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:04:30.289577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017334+0000","last_change":"2024-09-13T17:43:36.991160+0000","last_active":"2024-09-13T17:43:56.017334+0000","last_peered":"2024-09-13T17:43:56.017334+0000","last_clean":"2024-09-13T17:43:56.017334+0000","last_became_active":"2024-09-13T17:43:36.990271+0000","last_became_peered":"2024-09-13T17:43:36.990271+0000","last_unstale":"2024-09-13T17:43:56.017334+0000","last_undegraded":"2024-09-13T17:43:56.017334+0000","last_fullsized":"2024-09-13T17:43:56.017334+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:03.041558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013022+0000","last_change":"2024-09-13T17:43:36.612265+0000","last_active":"2024-09-13T17:43:56.013022+0000","last_peered":"2024-09-13T17:43:56.013022+0000","last_clean":"2024-09-13T17:43:56.013022+0000","last_became_active":"2024-09-13T17:43:36.609463+0000","last_became_peered":"2024-09-13T17:43:36.609463+0000","last_unstale":"2024-09-13T17:43:56.013022+0000","last_undegraded":"2024-09-13T17:43:56.013022+0000","last_fullsized":"2024-09-13T17:43:56.013022+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:51.123098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.021956+0000","last_change":"2024-09-13T17:43:55.661979+0000","last_active":"2024-09-13T17:43:56.021956+0000","last_peered":"2024-09-13T17:43:56.021956+0000","last_clean":"2024-09-13T17:43:56.021956+0000","last_became_active":"2024-09-13T17:43:55.661618+0000","last_became_peered":"2024-09-13T17:43:55.661618+0000","last_unstale":"2024-09-13T17:43:56.021956+0000","last_undegraded":"2024-09-13T17:43:56.021956+0000","last_fullsized":"2024-09-13T17:43:56.021956+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:55.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020077+0000","last_change":"2024-09-13T17:43:56.048692+0000","last_active":"2024-09-13T17:43:57.020077+0000","last_peered":"2024-09-13T17:43:57.020077+0000","last_clean":"2024-09-13T17:43:57.020077+0000","last_became_active":"2024-09-13T17:43:56.048428+0000","last_became_peered":"2024-09-13T17:43:56.048428+0000","last_unstale":"2024-09-13T17:43:57.020077+0000","last_undegraded":"2024-09-13T17:43:57.020077+0000","last_fullsized":"2024-09-13T17:43:57.020077+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:39:06.416508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022630+0000","last_change":"2024-09-13T17:43:36.994616+0000","last_active":"2024-09-13T17:43:56.022630+0000","last_peered":"2024-09-13T17:43:56.022630+0000","last_clean":"2024-09-13T17:43:56.022630+0000","last_became_active":"2024-09-13T17:43:36.994356+0000","last_became_peered":"2024-09-13T17:43:36.994356+0000","last_unstale":"2024-09-13T17:43:56.022630+0000","last_undegraded":"2024-09-13T17:43:56.022630+0000","last_fullsized":"2024-09-13T17:43:56.022630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014245+0000","last_change":"2024-09-13T17:43:36.601819+0000","last_active":"2024-09-13T17:43:56.014245+0000","last_peered":"2024-09-13T17:43:56.014245+0000","last_clean":"2024-09-13T17:43:56.014245+0000","last_became_active":"2024-09-13T17:43:36.601091+0000","last_became_peered":"2024-09-13T17:43:36.601091+0000","last_unstale":"2024-09-13T17:43:56.014245+0000","last_undegraded":"2024-09-13T17:43:56.014245+0000","last_fullsized":"2024-09-13T17:43:56.014245+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:22.130979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020772+0000","last_change":"2024-09-13T17:43:55.674696+0000","last_active":"2024-09-13T17:43:57.020772+0000","last_peered":"2024-09-13T17:43:57.020772+0000","last_clean":"2024-09-13T17:43:57.020772+0000","last_became_active":"2024-09-13T17:43:55.674339+0000","last_became_peered":"2024-09-13T17:43:55.674339+0000","last_unstale":"2024-09-13T17:43:57.020772+0000","last_undegraded":"2024-09-13T17:43:57.020772+0000","last_fullsized":"2024-09-13T17:43:57.020772+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:55.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527445+0000","last_change":"2024-09-13T17:43:56.040283+0000","last_active":"2024-09-13T17:43:57.527445+0000","last_peered":"2024-09-13T17:43:57.527445+0000","last_clean":"2024-09-13T17:43:57.527445+0000","last_became_active":"2024-09-13T17:43:56.039869+0000","last_became_peered":"2024-09-13T17:43:56.039869+0000","last_unstale":"2024-09-13T17:43:57.527445+0000","last_undegraded":"2024-09-13T17:43:57.527445+0000","last_fullsized":"2024-09-13T17:43:57.527445+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:14:18.322561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019552+0000","last_change":"2024-09-13T17:43:36.976449+0000","last_active":"2024-09-13T17:43:56.019552+0000","last_peered":"2024-09-13T17:43:56.019552+0000","last_clean":"2024-09-13T17:43:56.019552+0000","last_became_active":"2024-09-13T17:43:36.972460+0000","last_became_peered":"2024-09-13T17:43:36.972460+0000","last_unstale":"2024-09-13T17:43:56.019552+0000","last_undegraded":"2024-09-13T17:43:56.019552+0000","last_fullsized":"2024-09-13T17:43:56.019552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:55:12.422757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651120+0000","last_change":"2024-09-13T17:43:36.608221+0000","last_active":"2024-09-13T17:43:55.651120+0000","last_peered":"2024-09-13T17:43:55.651120+0000","last_clean":"2024-09-13T17:43:55.651120+0000","last_became_active":"2024-09-13T17:43:36.607779+0000","last_became_peered":"2024-09-13T17:43:36.607779+0000","last_unstale":"2024-09-13T17:43:55.651120+0000","last_undegraded":"2024-09-13T17:43:55.651120+0000","last_fullsized":"2024-09-13T17:43:55.651120+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:00.401036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526706+0000","last_change":"2024-09-13T17:43:55.676348+0000","last_active":"2024-09-13T17:43:57.526706+0000","last_peered":"2024-09-13T17:43:57.526706+0000","last_clean":"2024-09-13T17:43:57.526706+0000","last_became_active":"2024-09-13T17:43:55.675904+0000","last_became_peered":"2024-09-13T17:43:55.675904+0000","last_unstale":"2024-09-13T17:43:57.526706+0000","last_undegraded":"2024-09-13T17:43:57.526706+0000","last_fullsized":"2024-09-13T17:43:57.526706+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:31.669383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020695+0000","last_change":"2024-09-13T17:43:56.046636+0000","last_active":"2024-09-13T17:43:57.020695+0000","last_peered":"2024-09-13T17:43:57.020695+0000","last_clean":"2024-09-13T17:43:57.020695+0000","last_became_active":"2024-09-13T17:43:56.044857+0000","last_became_peered":"2024-09-13T17:43:56.044857+0000","last_unstale":"2024-09-13T17:43:57.020695+0000","last_undegraded":"2024-09-13T17:43:57.020695+0000","last_fullsized":"2024-09-13T17:43:57.020695+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.954905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.645714+0000","last_change":"2024-09-13T17:43:36.977858+0000","last_active":"2024-09-13T17:43:55.645714+0000","last_peered":"2024-09-13T17:43:55.645714+0000","last_clean":"2024-09-13T17:43:55.645714+0000","last_became_active":"2024-09-13T17:43:36.977282+0000","last_became_peered":"2024-09-13T17:43:36.977282+0000","last_unstale":"2024-09-13T17:43:55.645714+0000","last_undegraded":"2024-09-13T17:43:55.645714+0000","last_fullsized":"2024-09-13T17:43:55.645714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.642690+0000","last_change":"2024-09-13T17:43:36.601601+0000","last_active":"2024-09-13T17:43:55.642690+0000","last_peered":"2024-09-13T17:43:55.642690+0000","last_clean":"2024-09-13T17:43:55.642690+0000","last_became_active":"2024-09-13T17:43:36.594738+0000","last_became_peered":"2024-09-13T17:43:36.594738+0000","last_unstale":"2024-09-13T17:43:55.642690+0000","last_undegraded":"2024-09-13T17:43:55.642690+0000","last_fullsized":"2024-09-13T17:43:55.642690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:24:26.235235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527275+0000","last_change":"2024-09-13T17:43:55.676882+0000","last_active":"2024-09-13T17:43:57.527275+0000","last_peered":"2024-09-13T17:43:57.527275+0000","last_clean":"2024-09-13T17:43:57.527275+0000","last_became_active":"2024-09-13T17:43:55.676608+0000","last_became_peered":"2024-09-13T17:43:55.676608+0000","last_unstale":"2024-09-13T17:43:57.527275+0000","last_undegraded":"2024-09-13T17:43:57.527275+0000","last_fullsized":"2024-09-13T17:43:57.527275+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:02:05.943421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023194+0000","last_change":"2024-09-13T17:43:56.050364+0000","last_active":"2024-09-13T17:43:57.023194+0000","last_peered":"2024-09-13T17:43:57.023194+0000","last_clean":"2024-09-13T17:43:57.023194+0000","last_became_active":"2024-09-13T17:43:56.048480+0000","last_became_peered":"2024-09-13T17:43:56.048480+0000","last_unstale":"2024-09-13T17:43:57.023194+0000","last_undegraded":"2024-09-13T17:43:57.023194+0000","last_fullsized":"2024-09-13T17:43:57.023194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.761232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023521+0000","last_change":"2024-09-13T17:43:36.990024+0000","last_active":"2024-09-13T17:43:56.023521+0000","last_peered":"2024-09-13T17:43:56.023521+0000","last_clean":"2024-09-13T17:43:56.023521+0000","last_became_active":"2024-09-13T17:43:36.989774+0000","last_became_peered":"2024-09-13T17:43:36.989774+0000","last_unstale":"2024-09-13T17:43:56.023521+0000","last_undegraded":"2024-09-13T17:43:56.023521+0000","last_fullsized":"2024-09-13T17:43:56.023521+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:10.008585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012637+0000","last_change":"2024-09-13T17:43:36.608307+0000","last_active":"2024-09-13T17:43:56.012637+0000","last_peered":"2024-09-13T17:43:56.012637+0000","last_clean":"2024-09-13T17:43:56.012637+0000","last_became_active":"2024-09-13T17:43:36.607846+0000","last_became_peered":"2024-09-13T17:43:36.607846+0000","last_unstale":"2024-09-13T17:43:56.012637+0000","last_undegraded":"2024-09-13T17:43:56.012637+0000","last_fullsized":"2024-09-13T17:43:56.012637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:34.779723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.e","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527037+0000","last_change":"2024-09-13T17:43:55.676200+0000","last_active":"2024-09-13T17:43:57.527037+0000","last_peered":"2024-09-13T17:43:57.527037+0000","last_clean":"2024-09-13T17:43:57.527037+0000","last_became_active":"2024-09-13T17:43:55.675799+0000","last_became_peered":"2024-09-13T17:43:55.675799+0000","last_unstale":"2024-09-13T17:43:57.527037+0000","last_undegraded":"2024-09-13T17:43:57.527037+0000","last_fullsized":"2024-09-13T17:43:57.527037+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:56.957972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023013+0000","last_change":"2024-09-13T17:43:56.045732+0000","last_active":"2024-09-13T17:43:57.023013+0000","last_peered":"2024-09-13T17:43:57.023013+0000","last_clean":"2024-09-13T17:43:57.023013+0000","last_became_active":"2024-09-13T17:43:56.045135+0000","last_became_peered":"2024-09-13T17:43:56.045135+0000","last_unstale":"2024-09-13T17:43:57.023013+0000","last_undegraded":"2024-09-13T17:43:57.023013+0000","last_fullsized":"2024-09-13T17:43:57.023013+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:10:07.604989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013242+0000","last_change":"2024-09-13T17:43:36.999734+0000","last_active":"2024-09-13T17:43:56.013242+0000","last_peered":"2024-09-13T17:43:56.013242+0000","last_clean":"2024-09-13T17:43:56.013242+0000","last_became_active":"2024-09-13T17:43:36.999416+0000","last_became_peered":"2024-09-13T17:43:36.999416+0000","last_unstale":"2024-09-13T17:43:56.013242+0000","last_undegraded":"2024-09-13T17:43:56.013242+0000","last_fullsized":"2024-09-13T17:43:56.013242+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:57:11.465274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.020946+0000","last_change":"2024-09-13T17:43:36.598625+0000","last_active":"2024-09-13T17:43:56.020946+0000","last_peered":"2024-09-13T17:43:56.020946+0000","last_clean":"2024-09-13T17:43:56.020946+0000","last_became_active":"2024-09-13T17:43:36.598229+0000","last_became_peered":"2024-09-13T17:43:36.598229+0000","last_unstale":"2024-09-13T17:43:56.020946+0000","last_undegraded":"2024-09-13T17:43:56.020946+0000","last_fullsized":"2024-09-13T17:43:56.020946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:47:54.846315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"42'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.529568+0000","last_change":"2024-09-13T17:43:55.676805+0000","last_active":"2024-09-13T17:43:57.529568+0000","last_peered":"2024-09-13T17:43:57.529568+0000","last_clean":"2024-09-13T17:43:57.529568+0000","last_became_active":"2024-09-13T17:43:55.676468+0000","last_became_peered":"2024-09-13T17:43:55.676468+0000","last_unstale":"2024-09-13T17:43:57.529568+0000","last_undegraded":"2024-09-13T17:43:57.529568+0000","last_fullsized":"2024-09-13T17:43:57.529568+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:22.407289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.017942+0000","last_change":"2024-09-13T17:43:56.053288+0000","last_active":"2024-09-13T17:43:57.017942+0000","last_peered":"2024-09-13T17:43:57.017942+0000","last_clean":"2024-09-13T17:43:57.017942+0000","last_became_active":"2024-09-13T17:43:56.052585+0000","last_became_peered":"2024-09-13T17:43:56.052585+0000","last_unstale":"2024-09-13T17:43:57.017942+0000","last_undegraded":"2024-09-13T17:43:57.017942+0000","last_fullsized":"2024-09-13T17:43:57.017942+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:29.676452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015260+0000","last_change":"2024-09-13T17:43:36.972148+0000","last_active":"2024-09-13T17:43:56.015260+0000","last_peered":"2024-09-13T17:43:56.015260+0000","last_clean":"2024-09-13T17:43:56.015260+0000","last_became_active":"2024-09-13T17:43:36.971337+0000","last_became_peered":"2024-09-13T17:43:36.971337+0000","last_unstale":"2024-09-13T17:43:56.015260+0000","last_undegraded":"2024-09-13T17:43:56.015260+0000","last_fullsized":"2024-09-13T17:43:56.015260+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:15.285058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"36'2","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014845+0000","last_change":"2024-09-13T17:43:36.601639+0000","last_active":"2024-09-13T17:43:56.014845+0000","last_peered":"2024-09-13T17:43:56.014845+0000","last_clean":"2024-09-13T17:43:56.014845+0000","last_became_active":"2024-09-13T17:43:36.601329+0000","last_became_peered":"2024-09-13T17:43:36.601329+0000","last_unstale":"2024-09-13T17:43:56.014845+0000","last_undegraded":"2024-09-13T17:43:56.014845+0000","last_fullsized":"2024-09-13T17:43:56.014845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:43.542629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"38'130","reported_seq":283,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017230+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:43:56.017230+0000","last_peered":"2024-09-13T17:43:56.017230+0000","last_clean":"2024-09-13T17:43:56.017230+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:43:56.017230+0000","last_undegraded":"2024-09-13T17:43:56.017230+0000","last_fullsized":"2024-09-13T17:43:56.017230+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":130,"log_dups_size":0,"ondisk_log_size":130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":197,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521921+0000","last_change":"2024-09-13T17:43:55.676840+0000","last_active":"2024-09-13T17:43:57.521921+0000","last_peered":"2024-09-13T17:43:57.521921+0000","last_clean":"2024-09-13T17:43:57.521921+0000","last_became_active":"2024-09-13T17:43:55.676501+0000","last_became_peered":"2024-09-13T17:43:55.676501+0000","last_unstale":"2024-09-13T17:43:57.521921+0000","last_undegraded":"2024-09-13T17:43:57.521921+0000","last_fullsized":"2024-09-13T17:43:57.521921+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:47:52.080863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022963+0000","last_change":"2024-09-13T17:43:56.045890+0000","last_active":"2024-09-13T17:43:57.022963+0000","last_peered":"2024-09-13T17:43:57.022963+0000","last_clean":"2024-09-13T17:43:57.022963+0000","last_became_active":"2024-09-13T17:43:56.044886+0000","last_became_peered":"2024-09-13T17:43:56.044886+0000","last_unstale":"2024-09-13T17:43:57.022963+0000","last_undegraded":"2024-09-13T17:43:57.022963+0000","last_fullsized":"2024-09-13T17:43:57.022963+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:56:12.354007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.642583+0000","last_change":"2024-09-13T17:43:36.988557+0000","last_active":"2024-09-13T17:43:55.642583+0000","last_peered":"2024-09-13T17:43:55.642583+0000","last_clean":"2024-09-13T17:43:55.642583+0000","last_became_active":"2024-09-13T17:43:36.985705+0000","last_became_peered":"2024-09-13T17:43:36.985705+0000","last_unstale":"2024-09-13T17:43:55.642583+0000","last_undegraded":"2024-09-13T17:43:55.642583+0000","last_fullsized":"2024-09-13T17:43:55.642583+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017258+0000","last_change":"2024-09-13T17:43:36.614029+0000","last_active":"2024-09-13T17:43:56.017258+0000","last_peered":"2024-09-13T17:43:56.017258+0000","last_clean":"2024-09-13T17:43:56.017258+0000","last_became_active":"2024-09-13T17:43:36.612938+0000","last_became_peered":"2024-09-13T17:43:36.612938+0000","last_unstale":"2024-09-13T17:43:56.017258+0000","last_undegraded":"2024-09-13T17:43:56.017258+0000","last_fullsized":"2024-09-13T17:43:56.017258+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:57.173262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023168+0000","last_change":"2024-09-13T17:43:55.673716+0000","last_active":"2024-09-13T17:43:56.023168+0000","last_peered":"2024-09-13T17:43:56.023168+0000","last_clean":"2024-09-13T17:43:56.023168+0000","last_became_active":"2024-09-13T17:43:55.673446+0000","last_became_peered":"2024-09-13T17:43:55.673446+0000","last_unstale":"2024-09-13T17:43:56.023168+0000","last_undegraded":"2024-09-13T17:43:56.023168+0000","last_fullsized":"2024-09-13T17:43:56.023168+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:50.027712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524051+0000","last_change":"2024-09-13T17:43:56.055697+0000","last_active":"2024-09-13T17:43:57.524051+0000","last_peered":"2024-09-13T17:43:57.524051+0000","last_clean":"2024-09-13T17:43:57.524051+0000","last_became_active":"2024-09-13T17:43:56.053781+0000","last_became_peered":"2024-09-13T17:43:56.053781+0000","last_unstale":"2024-09-13T17:43:57.524051+0000","last_undegraded":"2024-09-13T17:43:57.524051+0000","last_fullsized":"2024-09-13T17:43:57.524051+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:07:56.060802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652975+0000","last_change":"2024-09-13T17:43:36.987379+0000","last_active":"2024-09-13T17:43:55.652975+0000","last_peered":"2024-09-13T17:43:55.652975+0000","last_clean":"2024-09-13T17:43:55.652975+0000","last_became_active":"2024-09-13T17:43:36.986990+0000","last_became_peered":"2024-09-13T17:43:36.986990+0000","last_unstale":"2024-09-13T17:43:55.652975+0000","last_undegraded":"2024-09-13T17:43:55.652975+0000","last_fullsized":"2024-09-13T17:43:55.652975+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652887+0000","last_change":"2024-09-13T17:43:36.596481+0000","last_active":"2024-09-13T17:43:55.652887+0000","last_peered":"2024-09-13T17:43:55.652887+0000","last_clean":"2024-09-13T17:43:55.652887+0000","last_became_active":"2024-09-13T17:43:36.594330+0000","last_became_peered":"2024-09-13T17:43:36.594330+0000","last_unstale":"2024-09-13T17:43:55.652887+0000","last_undegraded":"2024-09-13T17:43:55.652887+0000","last_fullsized":"2024-09-13T17:43:55.652887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:01:52.485090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019480+0000","last_change":"2024-09-13T17:43:55.671049+0000","last_active":"2024-09-13T17:43:57.019480+0000","last_peered":"2024-09-13T17:43:57.019480+0000","last_clean":"2024-09-13T17:43:57.019480+0000","last_became_active":"2024-09-13T17:43:55.670508+0000","last_became_peered":"2024-09-13T17:43:55.670508+0000","last_unstale":"2024-09-13T17:43:57.019480+0000","last_undegraded":"2024-09-13T17:43:57.019480+0000","last_fullsized":"2024-09-13T17:43:57.019480+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:30.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034358+0000","last_change":"2024-09-13T17:43:56.030662+0000","last_active":"2024-09-13T17:43:56.034358+0000","last_peered":"2024-09-13T17:43:56.034358+0000","last_clean":"2024-09-13T17:43:56.034358+0000","last_became_active":"2024-09-13T17:43:56.030409+0000","last_became_peered":"2024-09-13T17:43:56.030409+0000","last_unstale":"2024-09-13T17:43:56.034358+0000","last_undegraded":"2024-09-13T17:43:56.034358+0000","last_fullsized":"2024-09-13T17:43:56.034358+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:17.824302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026661+0000","last_change":"2024-09-13T17:43:36.994777+0000","last_active":"2024-09-13T17:43:56.026661+0000","last_peered":"2024-09-13T17:43:56.026661+0000","last_clean":"2024-09-13T17:43:56.026661+0000","last_became_active":"2024-09-13T17:43:36.994213+0000","last_became_peered":"2024-09-13T17:43:36.994213+0000","last_unstale":"2024-09-13T17:43:56.026661+0000","last_undegraded":"2024-09-13T17:43:56.026661+0000","last_fullsized":"2024-09-13T17:43:56.026661+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:55:50.585341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"36'1","reported_seq":16,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652076+0000","last_change":"2024-09-13T17:43:36.607688+0000","last_active":"2024-09-13T17:43:55.652076+0000","last_peered":"2024-09-13T17:43:55.652076+0000","last_clean":"2024-09-13T17:43:55.652076+0000","last_became_active":"2024-09-13T17:43:36.607290+0000","last_became_peered":"2024-09-13T17:43:36.607290+0000","last_unstale":"2024-09-13T17:43:55.652076+0000","last_undegraded":"2024-09-13T17:43:55.652076+0000","last_fullsized":"2024-09-13T17:43:55.652076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:20:24.428784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015245+0000","last_change":"2024-09-13T17:43:55.666279+0000","last_active":"2024-09-13T17:43:56.015245+0000","last_peered":"2024-09-13T17:43:56.015245+0000","last_clean":"2024-09-13T17:43:56.015245+0000","last_became_active":"2024-09-13T17:43:55.666007+0000","last_became_peered":"2024-09-13T17:43:55.666007+0000","last_unstale":"2024-09-13T17:43:56.015245+0000","last_undegraded":"2024-09-13T17:43:56.015245+0000","last_fullsized":"2024-09-13T17:43:56.015245+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:30.648352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522580+0000","last_change":"2024-09-13T17:43:56.036895+0000","last_active":"2024-09-13T17:43:57.522580+0000","last_peered":"2024-09-13T17:43:57.522580+0000","last_clean":"2024-09-13T17:43:57.522580+0000","last_became_active":"2024-09-13T17:43:56.035956+0000","last_became_peered":"2024-09-13T17:43:56.035956+0000","last_unstale":"2024-09-13T17:43:57.522580+0000","last_undegraded":"2024-09-13T17:43:57.522580+0000","last_fullsized":"2024-09-13T17:43:57.522580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:45:07.157910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017047+0000","last_change":"2024-09-13T17:43:36.996063+0000","last_active":"2024-09-13T17:43:56.017047+0000","last_peered":"2024-09-13T17:43:56.017047+0000","last_clean":"2024-09-13T17:43:56.017047+0000","last_became_active":"2024-09-13T17:43:36.995621+0000","last_became_peered":"2024-09-13T17:43:36.995621+0000","last_unstale":"2024-09-13T17:43:56.017047+0000","last_undegraded":"2024-09-13T17:43:56.017047+0000","last_fullsized":"2024-09-13T17:43:56.017047+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:33:12.598001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"36'2","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653138+0000","last_change":"2024-09-13T17:43:36.594362+0000","last_active":"2024-09-13T17:43:55.653138+0000","last_peered":"2024-09-13T17:43:55.653138+0000","last_clean":"2024-09-13T17:43:55.653138+0000","last_became_active":"2024-09-13T17:43:36.593113+0000","last_became_peered":"2024-09-13T17:43:36.593113+0000","last_unstale":"2024-09-13T17:43:55.653138+0000","last_undegraded":"2024-09-13T17:43:55.653138+0000","last_fullsized":"2024-09-13T17:43:55.653138+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:48:01.736566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017493+0000","last_change":"2024-09-13T17:43:55.666777+0000","last_active":"2024-09-13T17:43:56.017493+0000","last_peered":"2024-09-13T17:43:56.017493+0000","last_clean":"2024-09-13T17:43:56.017493+0000","last_became_active":"2024-09-13T17:43:55.666063+0000","last_became_peered":"2024-09-13T17:43:55.666063+0000","last_unstale":"2024-09-13T17:43:56.017493+0000","last_undegraded":"2024-09-13T17:43:56.017493+0000","last_fullsized":"2024-09-13T17:43:56.017493+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:14.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034293+0000","last_change":"2024-09-13T17:43:56.030128+0000","last_active":"2024-09-13T17:43:56.034293+0000","last_peered":"2024-09-13T17:43:56.034293+0000","last_clean":"2024-09-13T17:43:56.034293+0000","last_became_active":"2024-09-13T17:43:56.029794+0000","last_became_peered":"2024-09-13T17:43:56.029794+0000","last_unstale":"2024-09-13T17:43:56.034293+0000","last_undegraded":"2024-09-13T17:43:56.034293+0000","last_fullsized":"2024-09-13T17:43:56.034293+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:16:27.301097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026333+0000","last_change":"2024-09-13T17:43:36.996236+0000","last_active":"2024-09-13T17:43:56.026333+0000","last_peered":"2024-09-13T17:43:56.026333+0000","last_clean":"2024-09-13T17:43:56.026333+0000","last_became_active":"2024-09-13T17:43:36.995955+0000","last_became_peered":"2024-09-13T17:43:36.995955+0000","last_unstale":"2024-09-13T17:43:56.026333+0000","last_undegraded":"2024-09-13T17:43:56.026333+0000","last_fullsized":"2024-09-13T17:43:56.026333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:57:23.137553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.028670+0000","last_change":"2024-09-13T17:43:36.602229+0000","last_active":"2024-09-13T17:43:56.028670+0000","last_peered":"2024-09-13T17:43:56.028670+0000","last_clean":"2024-09-13T17:43:56.028670+0000","last_became_active":"2024-09-13T17:43:36.601122+0000","last_became_peered":"2024-09-13T17:43:36.601122+0000","last_unstale":"2024-09-13T17:43:56.028670+0000","last_undegraded":"2024-09-13T17:43:56.028670+0000","last_fullsized":"2024-09-13T17:43:56.028670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:27:25.802228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023043+0000","last_change":"2024-09-13T17:43:55.659694+0000","last_active":"2024-09-13T17:43:56.023043+0000","last_peered":"2024-09-13T17:43:56.023043+0000","last_clean":"2024-09-13T17:43:56.023043+0000","last_became_active":"2024-09-13T17:43:55.659358+0000","last_became_peered":"2024-09-13T17:43:55.659358+0000","last_unstale":"2024-09-13T17:43:56.023043+0000","last_undegraded":"2024-09-13T17:43:56.023043+0000","last_fullsized":"2024-09-13T17:43:56.023043+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:58.556852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.035445+0000","last_change":"2024-09-13T17:43:56.033294+0000","last_active":"2024-09-13T17:43:56.035445+0000","last_peered":"2024-09-13T17:43:56.035445+0000","last_clean":"2024-09-13T17:43:56.035445+0000","last_became_active":"2024-09-13T17:43:56.033034+0000","last_became_peered":"2024-09-13T17:43:56.033034+0000","last_unstale":"2024-09-13T17:43:56.035445+0000","last_undegraded":"2024-09-13T17:43:56.035445+0000","last_fullsized":"2024-09-13T17:43:56.035445+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:16.263521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"41'1","reported_seq":11,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.054080+0000","last_change":"2024-09-13T17:43:55.667221+0000","last_active":"2024-09-13T17:43:56.054080+0000","last_peered":"2024-09-13T17:43:56.054080+0000","last_clean":"2024-09-13T17:43:56.054080+0000","last_became_active":"2024-09-13T17:43:55.665780+0000","last_became_peered":"2024-09-13T17:43:55.665780+0000","last_unstale":"2024-09-13T17:43:56.054080+0000","last_undegraded":"2024-09-13T17:43:56.054080+0000","last_fullsized":"2024-09-13T17:43:56.054080+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:30.153604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023569+0000","last_change":"2024-09-13T17:43:36.967452+0000","last_active":"2024-09-13T17:43:56.023569+0000","last_peered":"2024-09-13T17:43:56.023569+0000","last_clean":"2024-09-13T17:43:56.023569+0000","last_became_active":"2024-09-13T17:43:36.966901+0000","last_became_peered":"2024-09-13T17:43:36.966901+0000","last_unstale":"2024-09-13T17:43:56.023569+0000","last_undegraded":"2024-09-13T17:43:56.023569+0000","last_fullsized":"2024-09-13T17:43:56.023569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:56.929802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"36'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.012720+0000","last_change":"2024-09-13T17:43:36.607932+0000","last_active":"2024-09-13T17:43:56.012720+0000","last_peered":"2024-09-13T17:43:56.012720+0000","last_clean":"2024-09-13T17:43:56.012720+0000","last_became_active":"2024-09-13T17:43:36.607651+0000","last_became_peered":"2024-09-13T17:43:36.607651+0000","last_unstale":"2024-09-13T17:43:56.012720+0000","last_undegraded":"2024-09-13T17:43:56.012720+0000","last_fullsized":"2024-09-13T17:43:56.012720+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:13:09.002010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526416+0000","last_change":"2024-09-13T17:43:56.046158+0000","last_active":"2024-09-13T17:43:57.526416+0000","last_peered":"2024-09-13T17:43:57.526416+0000","last_clean":"2024-09-13T17:43:57.526416+0000","last_became_active":"2024-09-13T17:43:56.041901+0000","last_became_peered":"2024-09-13T17:43:56.041901+0000","last_unstale":"2024-09-13T17:43:57.526416+0000","last_undegraded":"2024-09-13T17:43:57.526416+0000","last_fullsized":"2024-09-13T17:43:57.526416+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:26:40.596730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"41'4","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527303+0000","last_change":"2024-09-13T17:43:55.660775+0000","last_active":"2024-09-13T17:43:57.527303+0000","last_peered":"2024-09-13T17:43:57.527303+0000","last_clean":"2024-09-13T17:43:57.527303+0000","last_became_active":"2024-09-13T17:43:55.660354+0000","last_became_peered":"2024-09-13T17:43:55.660354+0000","last_unstale":"2024-09-13T17:43:57.527303+0000","last_undegraded":"2024-09-13T17:43:57.527303+0000","last_fullsized":"2024-09-13T17:43:57.527303+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:38:53.856576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.645660+0000","last_change":"2024-09-13T17:43:36.980350+0000","last_active":"2024-09-13T17:43:55.645660+0000","last_peered":"2024-09-13T17:43:55.645660+0000","last_clean":"2024-09-13T17:43:55.645660+0000","last_became_active":"2024-09-13T17:43:36.977778+0000","last_became_peered":"2024-09-13T17:43:36.977778+0000","last_unstale":"2024-09-13T17:43:55.645660+0000","last_undegraded":"2024-09-13T17:43:55.645660+0000","last_fullsized":"2024-09-13T17:43:55.645660+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023368+0000","last_change":"2024-09-13T17:43:36.597400+0000","last_active":"2024-09-13T17:43:56.023368+0000","last_peered":"2024-09-13T17:43:56.023368+0000","last_clean":"2024-09-13T17:43:56.023368+0000","last_became_active":"2024-09-13T17:43:36.596736+0000","last_became_peered":"2024-09-13T17:43:36.596736+0000","last_unstale":"2024-09-13T17:43:56.023368+0000","last_undegraded":"2024-09-13T17:43:56.023368+0000","last_fullsized":"2024-09-13T17:43:56.023368+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:21.792293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523976+0000","last_change":"2024-09-13T17:43:56.040260+0000","last_active":"2024-09-13T17:43:57.523976+0000","last_peered":"2024-09-13T17:43:57.523976+0000","last_clean":"2024-09-13T17:43:57.523976+0000","last_became_active":"2024-09-13T17:43:56.039593+0000","last_became_peered":"2024-09-13T17:43:56.039593+0000","last_unstale":"2024-09-13T17:43:57.523976+0000","last_undegraded":"2024-09-13T17:43:57.523976+0000","last_fullsized":"2024-09-13T17:43:57.523976+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:11.552387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019628+0000","last_change":"2024-09-13T17:43:36.976611+0000","last_active":"2024-09-13T17:43:56.019628+0000","last_peered":"2024-09-13T17:43:56.019628+0000","last_clean":"2024-09-13T17:43:56.019628+0000","last_became_active":"2024-09-13T17:43:36.972871+0000","last_became_peered":"2024-09-13T17:43:36.972871+0000","last_unstale":"2024-09-13T17:43:56.019628+0000","last_undegraded":"2024-09-13T17:43:56.019628+0000","last_fullsized":"2024-09-13T17:43:56.019628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:02.818469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013857+0000","last_change":"2024-09-13T17:43:36.602194+0000","last_active":"2024-09-13T17:43:56.013857+0000","last_peered":"2024-09-13T17:43:56.013857+0000","last_clean":"2024-09-13T17:43:56.013857+0000","last_became_active":"2024-09-13T17:43:36.601939+0000","last_became_peered":"2024-09-13T17:43:36.601939+0000","last_unstale":"2024-09-13T17:43:56.013857+0000","last_undegraded":"2024-09-13T17:43:56.013857+0000","last_fullsized":"2024-09-13T17:43:56.013857+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:32.706151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022308+0000","last_change":"2024-09-13T17:43:55.671328+0000","last_active":"2024-09-13T17:43:57.022308+0000","last_peered":"2024-09-13T17:43:57.022308+0000","last_clean":"2024-09-13T17:43:57.022308+0000","last_became_active":"2024-09-13T17:43:55.670859+0000","last_became_peered":"2024-09-13T17:43:55.670859+0000","last_unstale":"2024-09-13T17:43:57.022308+0000","last_undegraded":"2024-09-13T17:43:57.022308+0000","last_fullsized":"2024-09-13T17:43:57.022308+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:20:14.507116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526765+0000","last_change":"2024-09-13T17:43:56.041558+0000","last_active":"2024-09-13T17:43:57.526765+0000","last_peered":"2024-09-13T17:43:57.526765+0000","last_clean":"2024-09-13T17:43:57.526765+0000","last_became_active":"2024-09-13T17:43:56.041131+0000","last_became_peered":"2024-09-13T17:43:56.041131+0000","last_unstale":"2024-09-13T17:43:57.526765+0000","last_undegraded":"2024-09-13T17:43:57.526765+0000","last_fullsized":"2024-09-13T17:43:57.526765+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:31:41.351412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.651007+0000","last_change":"2024-09-13T17:43:36.988758+0000","last_active":"2024-09-13T17:43:55.651007+0000","last_peered":"2024-09-13T17:43:55.651007+0000","last_clean":"2024-09-13T17:43:55.651007+0000","last_became_active":"2024-09-13T17:43:36.987064+0000","last_became_peered":"2024-09-13T17:43:36.987064+0000","last_unstale":"2024-09-13T17:43:55.651007+0000","last_undegraded":"2024-09-13T17:43:55.651007+0000","last_fullsized":"2024-09-13T17:43:55.651007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"36'2","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.030955+0000","last_change":"2024-09-13T17:43:36.590347+0000","last_active":"2024-09-13T17:43:56.030955+0000","last_peered":"2024-09-13T17:43:56.030955+0000","last_clean":"2024-09-13T17:43:56.030955+0000","last_became_active":"2024-09-13T17:43:36.589982+0000","last_became_peered":"2024-09-13T17:43:36.589982+0000","last_unstale":"2024-09-13T17:43:56.030955+0000","last_undegraded":"2024-09-13T17:43:56.030955+0000","last_fullsized":"2024-09-13T17:43:56.030955+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:49:03.593342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"41'2","reported_seq":13,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.526922+0000","last_change":"2024-09-13T17:43:55.662879+0000","last_active":"2024-09-13T17:43:57.526922+0000","last_peered":"2024-09-13T17:43:57.526922+0000","last_clean":"2024-09-13T17:43:57.526922+0000","last_became_active":"2024-09-13T17:43:55.662566+0000","last_became_peered":"2024-09-13T17:43:55.662566+0000","last_unstale":"2024-09-13T17:43:57.526922+0000","last_undegraded":"2024-09-13T17:43:57.526922+0000","last_fullsized":"2024-09-13T17:43:57.526922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:45.840218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522284+0000","last_change":"2024-09-13T17:43:56.051810+0000","last_active":"2024-09-13T17:43:57.522284+0000","last_peered":"2024-09-13T17:43:57.522284+0000","last_clean":"2024-09-13T17:43:57.522284+0000","last_became_active":"2024-09-13T17:43:56.051201+0000","last_became_peered":"2024-09-13T17:43:56.051201+0000","last_unstale":"2024-09-13T17:43:57.522284+0000","last_undegraded":"2024-09-13T17:43:57.522284+0000","last_fullsized":"2024-09-13T17:43:57.522284+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:15.865568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024246+0000","last_change":"2024-09-13T17:43:36.991166+0000","last_active":"2024-09-13T17:43:56.024246+0000","last_peered":"2024-09-13T17:43:56.024246+0000","last_clean":"2024-09-13T17:43:56.024246+0000","last_became_active":"2024-09-13T17:43:36.990864+0000","last_became_peered":"2024-09-13T17:43:36.990864+0000","last_unstale":"2024-09-13T17:43:56.024246+0000","last_undegraded":"2024-09-13T17:43:56.024246+0000","last_fullsized":"2024-09-13T17:43:56.024246+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:32.362744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.14","version":"38'11","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023871+0000","last_change":"2024-09-13T17:43:36.603712+0000","last_active":"2024-09-13T17:43:56.023871+0000","last_peered":"2024-09-13T17:43:56.023871+0000","last_clean":"2024-09-13T17:43:56.023871+0000","last_became_active":"2024-09-13T17:43:36.603454+0000","last_became_peered":"2024-09-13T17:43:36.603454+0000","last_unstale":"2024-09-13T17:43:56.023871+0000","last_undegraded":"2024-09-13T17:43:56.023871+0000","last_fullsized":"2024-09-13T17:43:56.023871+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:01.349182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4196,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1a","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523554+0000","last_change":"2024-09-13T17:43:55.676053+0000","last_active":"2024-09-13T17:43:57.523554+0000","last_peered":"2024-09-13T17:43:57.523554+0000","last_clean":"2024-09-13T17:43:57.523554+0000","last_became_active":"2024-09-13T17:43:55.675758+0000","last_became_peered":"2024-09-13T17:43:55.675758+0000","last_unstale":"2024-09-13T17:43:57.523554+0000","last_undegraded":"2024-09-13T17:43:57.523554+0000","last_fullsized":"2024-09-13T17:43:57.523554+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:37.937247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.034212+0000","last_change":"2024-09-13T17:43:56.028847+0000","last_active":"2024-09-13T17:43:56.034212+0000","last_peered":"2024-09-13T17:43:56.034212+0000","last_clean":"2024-09-13T17:43:56.034212+0000","last_became_active":"2024-09-13T17:43:56.028496+0000","last_became_peered":"2024-09-13T17:43:56.028496+0000","last_unstale":"2024-09-13T17:43:56.034212+0000","last_undegraded":"2024-09-13T17:43:56.034212+0000","last_fullsized":"2024-09-13T17:43:56.034212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.090860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652649+0000","last_change":"2024-09-13T17:43:36.988822+0000","last_active":"2024-09-13T17:43:55.652649+0000","last_peered":"2024-09-13T17:43:55.652649+0000","last_clean":"2024-09-13T17:43:55.652649+0000","last_became_active":"2024-09-13T17:43:36.987189+0000","last_became_peered":"2024-09-13T17:43:36.987189+0000","last_unstale":"2024-09-13T17:43:55.652649+0000","last_undegraded":"2024-09-13T17:43:55.652649+0000","last_fullsized":"2024-09-13T17:43:55.652649+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:10.731418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.15","version":"36'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024766+0000","last_change":"2024-09-13T17:43:36.604426+0000","last_active":"2024-09-13T17:43:56.024766+0000","last_peered":"2024-09-13T17:43:56.024766+0000","last_clean":"2024-09-13T17:43:56.024766+0000","last_became_active":"2024-09-13T17:43:36.603659+0000","last_became_peered":"2024-09-13T17:43:36.603659+0000","last_unstale":"2024-09-13T17:43:56.024766+0000","last_undegraded":"2024-09-13T17:43:56.024766+0000","last_fullsized":"2024-09-13T17:43:56.024766+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:27.409650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1b","version":"41'1","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020106+0000","last_change":"2024-09-13T17:43:55.671013+0000","last_active":"2024-09-13T17:43:57.020106+0000","last_peered":"2024-09-13T17:43:57.020106+0000","last_clean":"2024-09-13T17:43:57.020106+0000","last_became_active":"2024-09-13T17:43:55.670751+0000","last_became_peered":"2024-09-13T17:43:55.670751+0000","last_unstale":"2024-09-13T17:43:57.020106+0000","last_undegraded":"2024-09-13T17:43:57.020106+0000","last_fullsized":"2024-09-13T17:43:57.020106+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:27:50.971922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019010+0000","last_change":"2024-09-13T17:43:56.048672+0000","last_active":"2024-09-13T17:43:57.019010+0000","last_peered":"2024-09-13T17:43:57.019010+0000","last_clean":"2024-09-13T17:43:57.019010+0000","last_became_active":"2024-09-13T17:43:56.048212+0000","last_became_peered":"2024-09-13T17:43:56.048212+0000","last_unstale":"2024-09-13T17:43:57.019010+0000","last_undegraded":"2024-09-13T17:43:57.019010+0000","last_fullsized":"2024-09-13T17:43:57.019010+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:04.391077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653121+0000","last_change":"2024-09-13T17:43:36.989522+0000","last_active":"2024-09-13T17:43:55.653121+0000","last_peered":"2024-09-13T17:43:55.653121+0000","last_clean":"2024-09-13T17:43:55.653121+0000","last_became_active":"2024-09-13T17:43:36.987659+0000","last_became_peered":"2024-09-13T17:43:36.987659+0000","last_unstale":"2024-09-13T17:43:55.653121+0000","last_undegraded":"2024-09-13T17:43:55.653121+0000","last_fullsized":"2024-09-13T17:43:55.653121+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014845+0000","last_change":"2024-09-13T17:43:36.610299+0000","last_active":"2024-09-13T17:43:56.014845+0000","last_peered":"2024-09-13T17:43:56.014845+0000","last_clean":"2024-09-13T17:43:56.014845+0000","last_became_active":"2024-09-13T17:43:36.609995+0000","last_became_peered":"2024-09-13T17:43:36.609995+0000","last_unstale":"2024-09-13T17:43:56.014845+0000","last_undegraded":"2024-09-13T17:43:56.014845+0000","last_fullsized":"2024-09-13T17:43:56.014845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:06.509485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522006+0000","last_change":"2024-09-13T17:43:55.660304+0000","last_active":"2024-09-13T17:43:57.522006+0000","last_peered":"2024-09-13T17:43:57.522006+0000","last_clean":"2024-09-13T17:43:57.522006+0000","last_became_active":"2024-09-13T17:43:55.658864+0000","last_became_peered":"2024-09-13T17:43:55.658864+0000","last_unstale":"2024-09-13T17:43:57.522006+0000","last_undegraded":"2024-09-13T17:43:57.522006+0000","last_fullsized":"2024-09-13T17:43:57.522006+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:17:53.802466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018659+0000","last_change":"2024-09-13T17:43:56.053153+0000","last_active":"2024-09-13T17:43:57.018659+0000","last_peered":"2024-09-13T17:43:57.018659+0000","last_clean":"2024-09-13T17:43:57.018659+0000","last_became_active":"2024-09-13T17:43:56.052913+0000","last_became_peered":"2024-09-13T17:43:56.052913+0000","last_unstale":"2024-09-13T17:43:57.018659+0000","last_undegraded":"2024-09-13T17:43:57.018659+0000","last_fullsized":"2024-09-13T17:43:57.018659+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:03:00.387308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652939+0000","last_change":"2024-09-13T17:43:36.994355+0000","last_active":"2024-09-13T17:43:55.652939+0000","last_peered":"2024-09-13T17:43:55.652939+0000","last_clean":"2024-09-13T17:43:55.652939+0000","last_became_active":"2024-09-13T17:43:36.988868+0000","last_became_peered":"2024-09-13T17:43:36.988868+0000","last_unstale":"2024-09-13T17:43:55.652939+0000","last_undegraded":"2024-09-13T17:43:55.652939+0000","last_fullsized":"2024-09-13T17:43:55.652939+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:52.129672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653442+0000","last_change":"2024-09-13T17:43:36.606733+0000","last_active":"2024-09-13T17:43:55.653442+0000","last_peered":"2024-09-13T17:43:55.653442+0000","last_clean":"2024-09-13T17:43:55.653442+0000","last_became_active":"2024-09-13T17:43:36.604421+0000","last_became_peered":"2024-09-13T17:43:36.604421+0000","last_unstale":"2024-09-13T17:43:55.653442+0000","last_undegraded":"2024-09-13T17:43:55.653442+0000","last_fullsized":"2024-09-13T17:43:55.653442+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:11:21.923719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022946+0000","last_change":"2024-09-13T17:43:55.662182+0000","last_active":"2024-09-13T17:43:56.022946+0000","last_peered":"2024-09-13T17:43:56.022946+0000","last_clean":"2024-09-13T17:43:56.022946+0000","last_became_active":"2024-09-13T17:43:55.661810+0000","last_became_peered":"2024-09-13T17:43:55.661810+0000","last_unstale":"2024-09-13T17:43:56.022946+0000","last_undegraded":"2024-09-13T17:43:56.022946+0000","last_fullsized":"2024-09-13T17:43:56.022946+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:48:21.091910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045655+0000","last_change":"2024-09-13T17:43:56.042895+0000","last_active":"2024-09-13T17:43:56.045655+0000","last_peered":"2024-09-13T17:43:56.045655+0000","last_clean":"2024-09-13T17:43:56.045655+0000","last_became_active":"2024-09-13T17:43:56.042430+0000","last_became_peered":"2024-09-13T17:43:56.042430+0000","last_unstale":"2024-09-13T17:43:56.045655+0000","last_undegraded":"2024-09-13T17:43:56.045655+0000","last_fullsized":"2024-09-13T17:43:56.045655+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.432317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"36'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013956+0000","last_change":"2024-09-13T17:43:36.612531+0000","last_active":"2024-09-13T17:43:56.013956+0000","last_peered":"2024-09-13T17:43:56.013956+0000","last_clean":"2024-09-13T17:43:56.013956+0000","last_became_active":"2024-09-13T17:43:36.610289+0000","last_became_peered":"2024-09-13T17:43:36.610289+0000","last_unstale":"2024-09-13T17:43:56.013956+0000","last_undegraded":"2024-09-13T17:43:56.013956+0000","last_fullsized":"2024-09-13T17:43:56.013956+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:35.554352+0000","last_clean_scrub_stamp":"2024-09-13T17:43:35.554352+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:47:09.724844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019699+0000","last_change":"2024-09-13T17:43:36.976787+0000","last_active":"2024-09-13T17:43:56.019699+0000","last_peered":"2024-09-13T17:43:56.019699+0000","last_clean":"2024-09-13T17:43:56.019699+0000","last_became_active":"2024-09-13T17:43:36.976205+0000","last_became_peered":"2024-09-13T17:43:36.976205+0000","last_unstale":"2024-09-13T17:43:56.019699+0000","last_undegraded":"2024-09-13T17:43:56.019699+0000","last_fullsized":"2024-09-13T17:43:56.019699+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:09.035544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"41'6","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524234+0000","last_change":"2024-09-13T17:43:55.676700+0000","last_active":"2024-09-13T17:43:57.524234+0000","last_peered":"2024-09-13T17:43:57.524234+0000","last_clean":"2024-09-13T17:43:57.524234+0000","last_became_active":"2024-09-13T17:43:55.676075+0000","last_became_peered":"2024-09-13T17:43:55.676075+0000","last_unstale":"2024-09-13T17:43:57.524234+0000","last_undegraded":"2024-09-13T17:43:57.524234+0000","last_fullsized":"2024-09-13T17:43:57.524234+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:54.611278+0000","last_clean_scrub_stamp":"2024-09-13T17:43:54.611278+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:02:59.379056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020569+0000","last_change":"2024-09-13T17:43:56.043210+0000","last_active":"2024-09-13T17:43:57.020569+0000","last_peered":"2024-09-13T17:43:57.020569+0000","last_clean":"2024-09-13T17:43:57.020569+0000","last_became_active":"2024-09-13T17:43:56.042583+0000","last_became_peered":"2024-09-13T17:43:56.042583+0000","last_unstale":"2024-09-13T17:43:57.020569+0000","last_undegraded":"2024-09-13T17:43:57.020569+0000","last_fullsized":"2024-09-13T17:43:57.020569+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:26:06.353192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022313+0000","last_change":"2024-09-13T17:43:36.990968+0000","last_active":"2024-09-13T17:43:56.022313+0000","last_peered":"2024-09-13T17:43:56.022313+0000","last_clean":"2024-09-13T17:43:56.022313+0000","last_became_active":"2024-09-13T17:43:36.990688+0000","last_became_peered":"2024-09-13T17:43:36.990688+0000","last_unstale":"2024-09-13T17:43:56.022313+0000","last_undegraded":"2024-09-13T17:43:56.022313+0000","last_fullsized":"2024-09-13T17:43:56.022313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022735+0000","last_change":"2024-09-13T17:43:56.048225+0000","last_active":"2024-09-13T17:43:57.022735+0000","last_peered":"2024-09-13T17:43:57.022735+0000","last_clean":"2024-09-13T17:43:57.022735+0000","last_became_active":"2024-09-13T17:43:56.047392+0000","last_became_peered":"2024-09-13T17:43:56.047392+0000","last_unstale":"2024-09-13T17:43:57.022735+0000","last_undegraded":"2024-09-13T17:43:57.022735+0000","last_fullsized":"2024-09-13T17:43:57.022735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:29:23.415541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.638887+0000","last_change":"2024-09-13T17:43:36.989315+0000","last_active":"2024-09-13T17:43:55.638887+0000","last_peered":"2024-09-13T17:43:55.638887+0000","last_clean":"2024-09-13T17:43:55.638887+0000","last_became_active":"2024-09-13T17:43:36.989070+0000","last_became_peered":"2024-09-13T17:43:36.989070+0000","last_unstale":"2024-09-13T17:43:55.638887+0000","last_undegraded":"2024-09-13T17:43:55.638887+0000","last_fullsized":"2024-09-13T17:43:55.638887+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.043071+0000","last_change":"2024-09-13T17:43:56.040467+0000","last_active":"2024-09-13T17:43:56.043071+0000","last_peered":"2024-09-13T17:43:56.043071+0000","last_clean":"2024-09-13T17:43:56.043071+0000","last_became_active":"2024-09-13T17:43:56.039885+0000","last_became_peered":"2024-09-13T17:43:56.039885+0000","last_unstale":"2024-09-13T17:43:56.043071+0000","last_undegraded":"2024-09-13T17:43:56.043071+0000","last_fullsized":"2024-09-13T17:43:56.043071+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:48.954905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024569+0000","last_change":"2024-09-13T17:43:36.967204+0000","last_active":"2024-09-13T17:43:56.024569+0000","last_peered":"2024-09-13T17:43:56.024569+0000","last_clean":"2024-09-13T17:43:56.024569+0000","last_became_active":"2024-09-13T17:43:36.966928+0000","last_became_peered":"2024-09-13T17:43:36.966928+0000","last_unstale":"2024-09-13T17:43:56.024569+0000","last_undegraded":"2024-09-13T17:43:56.024569+0000","last_fullsized":"2024-09-13T17:43:56.024569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:58.179836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020461+0000","last_change":"2024-09-13T17:43:56.043877+0000","last_active":"2024-09-13T17:43:57.020461+0000","last_peered":"2024-09-13T17:43:57.020461+0000","last_clean":"2024-09-13T17:43:57.020461+0000","last_became_active":"2024-09-13T17:43:56.043563+0000","last_became_peered":"2024-09-13T17:43:56.043563+0000","last_unstale":"2024-09-13T17:43:57.020461+0000","last_undegraded":"2024-09-13T17:43:57.020461+0000","last_fullsized":"2024-09-13T17:43:57.020461+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:39.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.019774+0000","last_change":"2024-09-13T17:43:36.998865+0000","last_active":"2024-09-13T17:43:56.019774+0000","last_peered":"2024-09-13T17:43:56.019774+0000","last_clean":"2024-09-13T17:43:56.019774+0000","last_became_active":"2024-09-13T17:43:36.998611+0000","last_became_peered":"2024-09-13T17:43:36.998611+0000","last_unstale":"2024-09-13T17:43:56.019774+0000","last_undegraded":"2024-09-13T17:43:56.019774+0000","last_fullsized":"2024-09-13T17:43:56.019774+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:31.977360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019616+0000","last_change":"2024-09-13T17:43:56.056341+0000","last_active":"2024-09-13T17:43:57.019616+0000","last_peered":"2024-09-13T17:43:57.019616+0000","last_clean":"2024-09-13T17:43:57.019616+0000","last_became_active":"2024-09-13T17:43:56.056095+0000","last_became_peered":"2024-09-13T17:43:56.056095+0000","last_unstale":"2024-09-13T17:43:57.019616+0000","last_undegraded":"2024-09-13T17:43:57.019616+0000","last_fullsized":"2024-09-13T17:43:57.019616+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:07.599159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024284+0000","last_change":"2024-09-13T17:43:36.994453+0000","last_active":"2024-09-13T17:43:56.024284+0000","last_peered":"2024-09-13T17:43:56.024284+0000","last_clean":"2024-09-13T17:43:56.024284+0000","last_became_active":"2024-09-13T17:43:36.993574+0000","last_became_peered":"2024-09-13T17:43:36.993574+0000","last_unstale":"2024-09-13T17:43:56.024284+0000","last_undegraded":"2024-09-13T17:43:56.024284+0000","last_fullsized":"2024-09-13T17:43:56.024284+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:17.039738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521775+0000","last_change":"2024-09-13T17:43:56.051980+0000","last_active":"2024-09-13T17:43:57.521775+0000","last_peered":"2024-09-13T17:43:57.521775+0000","last_clean":"2024-09-13T17:43:57.521775+0000","last_became_active":"2024-09-13T17:43:56.051475+0000","last_became_peered":"2024-09-13T17:43:56.051475+0000","last_unstale":"2024-09-13T17:43:57.521775+0000","last_undegraded":"2024-09-13T17:43:57.521775+0000","last_fullsized":"2024-09-13T17:43:57.521775+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:40:24.707236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015450+0000","last_change":"2024-09-13T17:43:36.999236+0000","last_active":"2024-09-13T17:43:56.015450+0000","last_peered":"2024-09-13T17:43:56.015450+0000","last_clean":"2024-09-13T17:43:56.015450+0000","last_became_active":"2024-09-13T17:43:36.998617+0000","last_became_peered":"2024-09-13T17:43:36.998617+0000","last_unstale":"2024-09-13T17:43:56.015450+0000","last_undegraded":"2024-09-13T17:43:56.015450+0000","last_fullsized":"2024-09-13T17:43:56.015450+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:17:43.213768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.042763+0000","last_change":"2024-09-13T17:43:56.039906+0000","last_active":"2024-09-13T17:43:56.042763+0000","last_peered":"2024-09-13T17:43:56.042763+0000","last_clean":"2024-09-13T17:43:56.042763+0000","last_became_active":"2024-09-13T17:43:56.039098+0000","last_became_peered":"2024-09-13T17:43:56.039098+0000","last_unstale":"2024-09-13T17:43:56.042763+0000","last_undegraded":"2024-09-13T17:43:56.042763+0000","last_fullsized":"2024-09-13T17:43:56.042763+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:33.996348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026414+0000","last_change":"2024-09-13T17:43:36.995992+0000","last_active":"2024-09-13T17:43:56.026414+0000","last_peered":"2024-09-13T17:43:56.026414+0000","last_clean":"2024-09-13T17:43:56.026414+0000","last_became_active":"2024-09-13T17:43:36.995839+0000","last_became_peered":"2024-09-13T17:43:36.995839+0000","last_unstale":"2024-09-13T17:43:56.026414+0000","last_undegraded":"2024-09-13T17:43:56.026414+0000","last_fullsized":"2024-09-13T17:43:56.026414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:03:43.285004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041609+0000","last_change":"2024-09-13T17:43:56.039634+0000","last_active":"2024-09-13T17:43:56.041609+0000","last_peered":"2024-09-13T17:43:56.041609+0000","last_clean":"2024-09-13T17:43:56.041609+0000","last_became_active":"2024-09-13T17:43:56.039393+0000","last_became_peered":"2024-09-13T17:43:56.039393+0000","last_unstale":"2024-09-13T17:43:56.041609+0000","last_undegraded":"2024-09-13T17:43:56.041609+0000","last_fullsized":"2024-09-13T17:43:56.041609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:02.407638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017201+0000","last_change":"2024-09-13T17:43:36.972052+0000","last_active":"2024-09-13T17:43:56.017201+0000","last_peered":"2024-09-13T17:43:56.017201+0000","last_clean":"2024-09-13T17:43:56.017201+0000","last_became_active":"2024-09-13T17:43:36.971235+0000","last_became_peered":"2024-09-13T17:43:36.971235+0000","last_unstale":"2024-09-13T17:43:56.017201+0000","last_undegraded":"2024-09-13T17:43:56.017201+0000","last_fullsized":"2024-09-13T17:43:56.017201+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:56:28.557515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.524273+0000","last_change":"2024-09-13T17:43:56.036290+0000","last_active":"2024-09-13T17:43:57.524273+0000","last_peered":"2024-09-13T17:43:57.524273+0000","last_clean":"2024-09-13T17:43:57.524273+0000","last_became_active":"2024-09-13T17:43:56.036007+0000","last_became_peered":"2024-09-13T17:43:56.036007+0000","last_unstale":"2024-09-13T17:43:57.524273+0000","last_undegraded":"2024-09-13T17:43:57.524273+0000","last_fullsized":"2024-09-13T17:43:57.524273+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:49:18.417056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022462+0000","last_change":"2024-09-13T17:43:36.989729+0000","last_active":"2024-09-13T17:43:56.022462+0000","last_peered":"2024-09-13T17:43:56.022462+0000","last_clean":"2024-09-13T17:43:56.022462+0000","last_became_active":"2024-09-13T17:43:36.989287+0000","last_became_peered":"2024-09-13T17:43:36.989287+0000","last_unstale":"2024-09-13T17:43:56.022462+0000","last_undegraded":"2024-09-13T17:43:56.022462+0000","last_fullsized":"2024-09-13T17:43:56.022462+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:18.160854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.023309+0000","last_change":"2024-09-13T17:43:56.050400+0000","last_active":"2024-09-13T17:43:57.023309+0000","last_peered":"2024-09-13T17:43:57.023309+0000","last_clean":"2024-09-13T17:43:57.023309+0000","last_became_active":"2024-09-13T17:43:56.048806+0000","last_became_peered":"2024-09-13T17:43:56.048806+0000","last_unstale":"2024-09-13T17:43:57.023309+0000","last_undegraded":"2024-09-13T17:43:57.023309+0000","last_fullsized":"2024-09-13T17:43:57.023309+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:51:19.183956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013483+0000","last_change":"2024-09-13T17:43:36.972925+0000","last_active":"2024-09-13T17:43:56.013483+0000","last_peered":"2024-09-13T17:43:56.013483+0000","last_clean":"2024-09-13T17:43:56.013483+0000","last_became_active":"2024-09-13T17:43:36.972388+0000","last_became_peered":"2024-09-13T17:43:36.972388+0000","last_unstale":"2024-09-13T17:43:56.013483+0000","last_undegraded":"2024-09-13T17:43:56.013483+0000","last_fullsized":"2024-09-13T17:43:56.013483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:30.822850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020051+0000","last_change":"2024-09-13T17:43:56.039613+0000","last_active":"2024-09-13T17:43:57.020051+0000","last_peered":"2024-09-13T17:43:57.020051+0000","last_clean":"2024-09-13T17:43:57.020051+0000","last_became_active":"2024-09-13T17:43:56.039255+0000","last_became_peered":"2024-09-13T17:43:56.039255+0000","last_unstale":"2024-09-13T17:43:57.020051+0000","last_undegraded":"2024-09-13T17:43:57.020051+0000","last_fullsized":"2024-09-13T17:43:57.020051+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:53:33.996348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014127+0000","last_change":"2024-09-13T17:43:36.995318+0000","last_active":"2024-09-13T17:43:56.014127+0000","last_peered":"2024-09-13T17:43:56.014127+0000","last_clean":"2024-09-13T17:43:56.014127+0000","last_became_active":"2024-09-13T17:43:36.995041+0000","last_became_peered":"2024-09-13T17:43:36.995041+0000","last_unstale":"2024-09-13T17:43:56.014127+0000","last_undegraded":"2024-09-13T17:43:56.014127+0000","last_fullsized":"2024-09-13T17:43:56.014127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:59.884186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523698+0000","last_change":"2024-09-13T17:43:56.034190+0000","last_active":"2024-09-13T17:43:57.523698+0000","last_peered":"2024-09-13T17:43:57.523698+0000","last_clean":"2024-09-13T17:43:57.523698+0000","last_became_active":"2024-09-13T17:43:56.031183+0000","last_became_peered":"2024-09-13T17:43:56.031183+0000","last_unstale":"2024-09-13T17:43:57.523698+0000","last_undegraded":"2024-09-13T17:43:57.523698+0000","last_fullsized":"2024-09-13T17:43:57.523698+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:07:08.067994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.022710+0000","last_change":"2024-09-13T17:43:36.981632+0000","last_active":"2024-09-13T17:43:56.022710+0000","last_peered":"2024-09-13T17:43:56.022710+0000","last_clean":"2024-09-13T17:43:56.022710+0000","last_became_active":"2024-09-13T17:43:36.977775+0000","last_became_peered":"2024-09-13T17:43:36.977775+0000","last_unstale":"2024-09-13T17:43:56.022710+0000","last_undegraded":"2024-09-13T17:43:56.022710+0000","last_fullsized":"2024-09-13T17:43:56.022710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:44.549204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.056493+0000","last_change":"2024-09-13T17:43:56.055026+0000","last_active":"2024-09-13T17:43:56.056493+0000","last_peered":"2024-09-13T17:43:56.056493+0000","last_clean":"2024-09-13T17:43:56.056493+0000","last_became_active":"2024-09-13T17:43:56.054419+0000","last_became_peered":"2024-09-13T17:43:56.054419+0000","last_unstale":"2024-09-13T17:43:56.056493+0000","last_undegraded":"2024-09-13T17:43:56.056493+0000","last_fullsized":"2024-09-13T17:43:56.056493+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:28:04.391077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024362+0000","last_change":"2024-09-13T17:43:36.976924+0000","last_active":"2024-09-13T17:43:56.024362+0000","last_peered":"2024-09-13T17:43:56.024362+0000","last_clean":"2024-09-13T17:43:56.024362+0000","last_became_active":"2024-09-13T17:43:36.975597+0000","last_became_peered":"2024-09-13T17:43:36.975597+0000","last_unstale":"2024-09-13T17:43:56.024362+0000","last_undegraded":"2024-09-13T17:43:56.024362+0000","last_fullsized":"2024-09-13T17:43:56.024362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:10:43.031842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019796+0000","last_change":"2024-09-13T17:43:56.046366+0000","last_active":"2024-09-13T17:43:57.019796+0000","last_peered":"2024-09-13T17:43:57.019796+0000","last_clean":"2024-09-13T17:43:57.019796+0000","last_became_active":"2024-09-13T17:43:56.044770+0000","last_became_peered":"2024-09-13T17:43:56.044770+0000","last_unstale":"2024-09-13T17:43:57.019796+0000","last_undegraded":"2024-09-13T17:43:57.019796+0000","last_fullsized":"2024-09-13T17:43:57.019796+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:51:02.407638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015193+0000","last_change":"2024-09-13T17:43:36.996728+0000","last_active":"2024-09-13T17:43:56.015193+0000","last_peered":"2024-09-13T17:43:56.015193+0000","last_clean":"2024-09-13T17:43:56.015193+0000","last_became_active":"2024-09-13T17:43:36.996468+0000","last_became_peered":"2024-09-13T17:43:36.996468+0000","last_unstale":"2024-09-13T17:43:56.015193+0000","last_undegraded":"2024-09-13T17:43:56.015193+0000","last_fullsized":"2024-09-13T17:43:56.015193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:19:56.456764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.522395+0000","last_change":"2024-09-13T17:43:56.040993+0000","last_active":"2024-09-13T17:43:57.522395+0000","last_peered":"2024-09-13T17:43:57.522395+0000","last_clean":"2024-09-13T17:43:57.522395+0000","last_became_active":"2024-09-13T17:43:56.037505+0000","last_became_peered":"2024-09-13T17:43:56.037505+0000","last_unstale":"2024-09-13T17:43:57.522395+0000","last_undegraded":"2024-09-13T17:43:57.522395+0000","last_fullsized":"2024-09-13T17:43:57.522395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:33:26.680526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.652679+0000","last_change":"2024-09-13T17:43:36.991727+0000","last_active":"2024-09-13T17:43:55.652679+0000","last_peered":"2024-09-13T17:43:55.652679+0000","last_clean":"2024-09-13T17:43:55.652679+0000","last_became_active":"2024-09-13T17:43:36.988498+0000","last_became_peered":"2024-09-13T17:43:36.988498+0000","last_unstale":"2024-09-13T17:43:55.652679+0000","last_undegraded":"2024-09-13T17:43:55.652679+0000","last_fullsized":"2024-09-13T17:43:55.652679+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:55:21.237157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018961+0000","last_change":"2024-09-13T17:43:56.044757+0000","last_active":"2024-09-13T17:43:57.018961+0000","last_peered":"2024-09-13T17:43:57.018961+0000","last_clean":"2024-09-13T17:43:57.018961+0000","last_became_active":"2024-09-13T17:43:56.044378+0000","last_became_peered":"2024-09-13T17:43:56.044378+0000","last_unstale":"2024-09-13T17:43:57.018961+0000","last_undegraded":"2024-09-13T17:43:57.018961+0000","last_fullsized":"2024-09-13T17:43:57.018961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:24:33.432317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.029101+0000","last_change":"2024-09-13T17:43:36.976795+0000","last_active":"2024-09-13T17:43:56.029101+0000","last_peered":"2024-09-13T17:43:56.029101+0000","last_clean":"2024-09-13T17:43:56.029101+0000","last_became_active":"2024-09-13T17:43:36.974036+0000","last_became_peered":"2024-09-13T17:43:36.974036+0000","last_unstale":"2024-09-13T17:43:56.029101+0000","last_undegraded":"2024-09-13T17:43:56.029101+0000","last_fullsized":"2024-09-13T17:43:56.029101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:08:35.786298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020905+0000","last_change":"2024-09-13T17:43:56.046721+0000","last_active":"2024-09-13T17:43:57.020905+0000","last_peered":"2024-09-13T17:43:57.020905+0000","last_clean":"2024-09-13T17:43:57.020905+0000","last_became_active":"2024-09-13T17:43:56.045178+0000","last_became_peered":"2024-09-13T17:43:56.045178+0000","last_unstale":"2024-09-13T17:43:57.020905+0000","last_undegraded":"2024-09-13T17:43:57.020905+0000","last_fullsized":"2024-09-13T17:43:57.020905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:38:17.824302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017410+0000","last_change":"2024-09-13T17:43:36.997635+0000","last_active":"2024-09-13T17:43:56.017410+0000","last_peered":"2024-09-13T17:43:56.017410+0000","last_clean":"2024-09-13T17:43:56.017410+0000","last_became_active":"2024-09-13T17:43:36.997014+0000","last_became_peered":"2024-09-13T17:43:36.997014+0000","last_unstale":"2024-09-13T17:43:56.017410+0000","last_undegraded":"2024-09-13T17:43:56.017410+0000","last_fullsized":"2024-09-13T17:43:56.017410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:29:36.971585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.022460+0000","last_change":"2024-09-13T17:43:56.045478+0000","last_active":"2024-09-13T17:43:57.022460+0000","last_peered":"2024-09-13T17:43:57.022460+0000","last_clean":"2024-09-13T17:43:57.022460+0000","last_became_active":"2024-09-13T17:43:56.044526+0000","last_became_peered":"2024-09-13T17:43:56.044526+0000","last_unstale":"2024-09-13T17:43:57.022460+0000","last_undegraded":"2024-09-13T17:43:57.022460+0000","last_fullsized":"2024-09-13T17:43:57.022460+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:06:38.945499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023632+0000","last_change":"2024-09-13T17:43:36.989971+0000","last_active":"2024-09-13T17:43:56.023632+0000","last_peered":"2024-09-13T17:43:56.023632+0000","last_clean":"2024-09-13T17:43:56.023632+0000","last_became_active":"2024-09-13T17:43:36.989598+0000","last_became_peered":"2024-09-13T17:43:36.989598+0000","last_unstale":"2024-09-13T17:43:56.023632+0000","last_undegraded":"2024-09-13T17:43:56.023632+0000","last_fullsized":"2024-09-13T17:43:56.023632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:14:07.516060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.527194+0000","last_change":"2024-09-13T17:43:56.036448+0000","last_active":"2024-09-13T17:43:57.527194+0000","last_peered":"2024-09-13T17:43:57.527194+0000","last_clean":"2024-09-13T17:43:57.527194+0000","last_became_active":"2024-09-13T17:43:56.035769+0000","last_became_peered":"2024-09-13T17:43:56.035769+0000","last_unstale":"2024-09-13T17:43:57.527194+0000","last_undegraded":"2024-09-13T17:43:57.527194+0000","last_fullsized":"2024-09-13T17:43:57.527194+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:07.252720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014211+0000","last_change":"2024-09-13T17:43:36.998495+0000","last_active":"2024-09-13T17:43:56.014211+0000","last_peered":"2024-09-13T17:43:56.014211+0000","last_clean":"2024-09-13T17:43:56.014211+0000","last_became_active":"2024-09-13T17:43:36.998203+0000","last_became_peered":"2024-09-13T17:43:36.998203+0000","last_unstale":"2024-09-13T17:43:56.014211+0000","last_undegraded":"2024-09-13T17:43:56.014211+0000","last_fullsized":"2024-09-13T17:43:56.014211+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:03:39.769029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020314+0000","last_change":"2024-09-13T17:43:56.046084+0000","last_active":"2024-09-13T17:43:57.020314+0000","last_peered":"2024-09-13T17:43:57.020314+0000","last_clean":"2024-09-13T17:43:57.020314+0000","last_became_active":"2024-09-13T17:43:56.044328+0000","last_became_peered":"2024-09-13T17:43:56.044328+0000","last_unstale":"2024-09-13T17:43:57.020314+0000","last_undegraded":"2024-09-13T17:43:57.020314+0000","last_fullsized":"2024-09-13T17:43:57.020314+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:23:16.263521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.023142+0000","last_change":"2024-09-13T17:43:36.989784+0000","last_active":"2024-09-13T17:43:56.023142+0000","last_peered":"2024-09-13T17:43:56.023142+0000","last_clean":"2024-09-13T17:43:56.023142+0000","last_became_active":"2024-09-13T17:43:36.989513+0000","last_became_peered":"2024-09-13T17:43:36.989513+0000","last_unstale":"2024-09-13T17:43:56.023142+0000","last_undegraded":"2024-09-13T17:43:56.023142+0000","last_fullsized":"2024-09-13T17:43:56.023142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:36:24.094337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521734+0000","last_change":"2024-09-13T17:43:56.038766+0000","last_active":"2024-09-13T17:43:57.521734+0000","last_peered":"2024-09-13T17:43:57.521734+0000","last_clean":"2024-09-13T17:43:57.521734+0000","last_became_active":"2024-09-13T17:43:56.038508+0000","last_became_peered":"2024-09-13T17:43:56.038508+0000","last_unstale":"2024-09-13T17:43:57.521734+0000","last_undegraded":"2024-09-13T17:43:57.521734+0000","last_fullsized":"2024-09-13T17:43:57.521734+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:18:42.670857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.024443+0000","last_change":"2024-09-13T17:43:36.970077+0000","last_active":"2024-09-13T17:43:56.024443+0000","last_peered":"2024-09-13T17:43:56.024443+0000","last_clean":"2024-09-13T17:43:56.024443+0000","last_became_active":"2024-09-13T17:43:36.969731+0000","last_became_peered":"2024-09-13T17:43:36.969731+0000","last_unstale":"2024-09-13T17:43:56.024443+0000","last_undegraded":"2024-09-13T17:43:56.024443+0000","last_fullsized":"2024-09-13T17:43:56.024443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:33.712725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.020251+0000","last_change":"2024-09-13T17:43:56.048654+0000","last_active":"2024-09-13T17:43:57.020251+0000","last_peered":"2024-09-13T17:43:57.020251+0000","last_clean":"2024-09-13T17:43:57.020251+0000","last_became_active":"2024-09-13T17:43:56.048293+0000","last_became_peered":"2024-09-13T17:43:56.048293+0000","last_unstale":"2024-09-13T17:43:57.020251+0000","last_undegraded":"2024-09-13T17:43:57.020251+0000","last_fullsized":"2024-09-13T17:43:57.020251+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:10:49.090860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015522+0000","last_change":"2024-09-13T17:43:36.999259+0000","last_active":"2024-09-13T17:43:56.015522+0000","last_peered":"2024-09-13T17:43:56.015522+0000","last_clean":"2024-09-13T17:43:56.015522+0000","last_became_active":"2024-09-13T17:43:36.998970+0000","last_became_peered":"2024-09-13T17:43:36.998970+0000","last_unstale":"2024-09-13T17:43:56.015522+0000","last_undegraded":"2024-09-13T17:43:56.015522+0000","last_fullsized":"2024-09-13T17:43:56.015522+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:57:26.856441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.018373+0000","last_change":"2024-09-13T17:43:56.044870+0000","last_active":"2024-09-13T17:43:57.018373+0000","last_peered":"2024-09-13T17:43:57.018373+0000","last_clean":"2024-09-13T17:43:57.018373+0000","last_became_active":"2024-09-13T17:43:56.044589+0000","last_became_peered":"2024-09-13T17:43:56.044589+0000","last_unstale":"2024-09-13T17:43:57.018373+0000","last_undegraded":"2024-09-13T17:43:57.018373+0000","last_fullsized":"2024-09-13T17:43:57.018373+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:16:27.301097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.015422+0000","last_change":"2024-09-13T17:43:36.972143+0000","last_active":"2024-09-13T17:43:56.015422+0000","last_peered":"2024-09-13T17:43:56.015422+0000","last_clean":"2024-09-13T17:43:56.015422+0000","last_became_active":"2024-09-13T17:43:36.971262+0000","last_became_peered":"2024-09-13T17:43:36.971262+0000","last_unstale":"2024-09-13T17:43:56.015422+0000","last_undegraded":"2024-09-13T17:43:56.015422+0000","last_fullsized":"2024-09-13T17:43:56.015422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:31:24.359313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.521001+0000","last_change":"2024-09-13T17:43:56.031502+0000","last_active":"2024-09-13T17:43:57.521001+0000","last_peered":"2024-09-13T17:43:57.521001+0000","last_clean":"2024-09-13T17:43:57.521001+0000","last_became_active":"2024-09-13T17:43:56.029129+0000","last_became_peered":"2024-09-13T17:43:56.029129+0000","last_unstale":"2024-09-13T17:43:57.521001+0000","last_undegraded":"2024-09-13T17:43:57.521001+0000","last_fullsized":"2024-09-13T17:43:57.521001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:50:26.414401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017282+0000","last_change":"2024-09-13T17:43:36.998418+0000","last_active":"2024-09-13T17:43:56.017282+0000","last_peered":"2024-09-13T17:43:56.017282+0000","last_clean":"2024-09-13T17:43:56.017282+0000","last_became_active":"2024-09-13T17:43:36.998145+0000","last_became_peered":"2024-09-13T17:43:36.998145+0000","last_unstale":"2024-09-13T17:43:56.017282+0000","last_undegraded":"2024-09-13T17:43:56.017282+0000","last_fullsized":"2024-09-13T17:43:56.017282+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:32:12.103252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.046015+0000","last_change":"2024-09-13T17:43:56.042674+0000","last_active":"2024-09-13T17:43:56.046015+0000","last_peered":"2024-09-13T17:43:56.046015+0000","last_clean":"2024-09-13T17:43:56.046015+0000","last_became_active":"2024-09-13T17:43:56.042401+0000","last_became_peered":"2024-09-13T17:43:56.042401+0000","last_unstale":"2024-09-13T17:43:56.046015+0000","last_undegraded":"2024-09-13T17:43:56.046015+0000","last_fullsized":"2024-09-13T17:43:56.046015+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:56:29.676452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017585+0000","last_change":"2024-09-13T17:43:36.997792+0000","last_active":"2024-09-13T17:43:56.017585+0000","last_peered":"2024-09-13T17:43:56.017585+0000","last_clean":"2024-09-13T17:43:56.017585+0000","last_became_active":"2024-09-13T17:43:36.997361+0000","last_became_peered":"2024-09-13T17:43:36.997361+0000","last_unstale":"2024-09-13T17:43:56.017585+0000","last_undegraded":"2024-09-13T17:43:56.017585+0000","last_fullsized":"2024-09-13T17:43:56.017585+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:18:40.506501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.041600+0000","last_change":"2024-09-13T17:43:56.038807+0000","last_active":"2024-09-13T17:43:56.041600+0000","last_peered":"2024-09-13T17:43:56.041600+0000","last_clean":"2024-09-13T17:43:56.041600+0000","last_became_active":"2024-09-13T17:43:56.038214+0000","last_became_peered":"2024-09-13T17:43:56.038214+0000","last_unstale":"2024-09-13T17:43:56.041600+0000","last_undegraded":"2024-09-13T17:43:56.041600+0000","last_fullsized":"2024-09-13T17:43:56.041600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:48:07.599159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013167+0000","last_change":"2024-09-13T17:43:36.999611+0000","last_active":"2024-09-13T17:43:56.013167+0000","last_peered":"2024-09-13T17:43:56.013167+0000","last_clean":"2024-09-13T17:43:56.013167+0000","last_became_active":"2024-09-13T17:43:36.998996+0000","last_became_peered":"2024-09-13T17:43:36.998996+0000","last_unstale":"2024-09-13T17:43:56.013167+0000","last_undegraded":"2024-09-13T17:43:56.013167+0000","last_fullsized":"2024-09-13T17:43:56.013167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:40:40.880802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.045463+0000","last_change":"2024-09-13T17:43:56.040629+0000","last_active":"2024-09-13T17:43:56.045463+0000","last_peered":"2024-09-13T17:43:56.045463+0000","last_clean":"2024-09-13T17:43:56.045463+0000","last_became_active":"2024-09-13T17:43:56.040236+0000","last_became_peered":"2024-09-13T17:43:56.040236+0000","last_unstale":"2024-09-13T17:43:56.045463+0000","last_undegraded":"2024-09-13T17:43:56.045463+0000","last_fullsized":"2024-09-13T17:43:56.045463+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:20:39.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.014344+0000","last_change":"2024-09-13T17:43:36.996807+0000","last_active":"2024-09-13T17:43:56.014344+0000","last_peered":"2024-09-13T17:43:56.014344+0000","last_clean":"2024-09-13T17:43:56.014344+0000","last_became_active":"2024-09-13T17:43:36.996515+0000","last_became_peered":"2024-09-13T17:43:36.996515+0000","last_unstale":"2024-09-13T17:43:56.014344+0000","last_undegraded":"2024-09-13T17:43:56.014344+0000","last_fullsized":"2024-09-13T17:43:56.014344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:46.786049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.035419+0000","last_change":"2024-09-13T17:43:56.033234+0000","last_active":"2024-09-13T17:43:56.035419+0000","last_peered":"2024-09-13T17:43:56.035419+0000","last_clean":"2024-09-13T17:43:56.035419+0000","last_became_active":"2024-09-13T17:43:56.032969+0000","last_became_peered":"2024-09-13T17:43:56.032969+0000","last_unstale":"2024-09-13T17:43:56.035419+0000","last_undegraded":"2024-09-13T17:43:56.035419+0000","last_fullsized":"2024-09-13T17:43:56.035419+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:08:43.761232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.020435+0000","last_change":"2024-09-13T17:43:36.998726+0000","last_active":"2024-09-13T17:43:56.020435+0000","last_peered":"2024-09-13T17:43:56.020435+0000","last_clean":"2024-09-13T17:43:56.020435+0000","last_became_active":"2024-09-13T17:43:36.998283+0000","last_became_peered":"2024-09-13T17:43:36.998283+0000","last_unstale":"2024-09-13T17:43:56.020435+0000","last_undegraded":"2024-09-13T17:43:56.020435+0000","last_fullsized":"2024-09-13T17:43:56.020435+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:02:03.727034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.019395+0000","last_change":"2024-09-13T17:43:56.056236+0000","last_active":"2024-09-13T17:43:57.019395+0000","last_peered":"2024-09-13T17:43:57.019395+0000","last_clean":"2024-09-13T17:43:57.019395+0000","last_became_active":"2024-09-13T17:43:56.055988+0000","last_became_peered":"2024-09-13T17:43:56.055988+0000","last_unstale":"2024-09-13T17:43:57.019395+0000","last_undegraded":"2024-09-13T17:43:57.019395+0000","last_fullsized":"2024-09-13T17:43:57.019395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:30:57.966224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.013619+0000","last_change":"2024-09-13T17:43:36.972979+0000","last_active":"2024-09-13T17:43:56.013619+0000","last_peered":"2024-09-13T17:43:56.013619+0000","last_clean":"2024-09-13T17:43:56.013619+0000","last_became_active":"2024-09-13T17:43:36.972625+0000","last_became_peered":"2024-09-13T17:43:36.972625+0000","last_unstale":"2024-09-13T17:43:56.013619+0000","last_undegraded":"2024-09-13T17:43:56.013619+0000","last_fullsized":"2024-09-13T17:43:56.013619+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:20:52.901570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.523282+0000","last_change":"2024-09-13T17:43:56.052186+0000","last_active":"2024-09-13T17:43:57.523282+0000","last_peered":"2024-09-13T17:43:57.523282+0000","last_clean":"2024-09-13T17:43:57.523282+0000","last_became_active":"2024-09-13T17:43:56.051554+0000","last_became_peered":"2024-09-13T17:43:56.051554+0000","last_unstale":"2024-09-13T17:43:57.523282+0000","last_undegraded":"2024-09-13T17:43:57.523282+0000","last_fullsized":"2024-09-13T17:43:57.523282+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:36:51.786165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-13T17:43:55.653359+0000","last_change":"2024-09-13T17:43:36.988922+0000","last_active":"2024-09-13T17:43:55.653359+0000","last_peered":"2024-09-13T17:43:55.653359+0000","last_clean":"2024-09-13T17:43:55.653359+0000","last_became_active":"2024-09-13T17:43:36.988648+0000","last_became_peered":"2024-09-13T17:43:36.988648+0000","last_unstale":"2024-09-13T17:43:55.653359+0000","last_undegraded":"2024-09-13T17:43:55.653359+0000","last_fullsized":"2024-09-13T17:43:55.653359+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:16:23.352799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.520682+0000","last_change":"2024-09-13T17:43:56.041845+0000","last_active":"2024-09-13T17:43:57.520682+0000","last_peered":"2024-09-13T17:43:57.520682+0000","last_clean":"2024-09-13T17:43:57.520682+0000","last_became_active":"2024-09-13T17:43:56.035755+0000","last_became_peered":"2024-09-13T17:43:56.035755+0000","last_unstale":"2024-09-13T17:43:57.520682+0000","last_undegraded":"2024-09-13T17:43:57.520682+0000","last_fullsized":"2024-09-13T17:43:57.520682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:06:58.808502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.026496+0000","last_change":"2024-09-13T17:43:36.974086+0000","last_active":"2024-09-13T17:43:56.026496+0000","last_peered":"2024-09-13T17:43:56.026496+0000","last_clean":"2024-09-13T17:43:56.026496+0000","last_became_active":"2024-09-13T17:43:36.973536+0000","last_became_peered":"2024-09-13T17:43:36.973536+0000","last_unstale":"2024-09-13T17:43:56.026496+0000","last_undegraded":"2024-09-13T17:43:56.026496+0000","last_fullsized":"2024-09-13T17:43:56.026496+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:22:24.478487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-13T17:43:57.520997+0000","last_change":"2024-09-13T17:43:56.051373+0000","last_active":"2024-09-13T17:43:57.520997+0000","last_peered":"2024-09-13T17:43:57.520997+0000","last_clean":"2024-09-13T17:43:57.520997+0000","last_became_active":"2024-09-13T17:43:56.045228+0000","last_became_peered":"2024-09-13T17:43:56.045228+0000","last_unstale":"2024-09-13T17:43:57.520997+0000","last_undegraded":"2024-09-13T17:43:57.520997+0000","last_fullsized":"2024-09-13T17:43:57.520997+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:55.619530+0000","last_clean_scrub_stamp":"2024-09-13T17:43:55.619530+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:49:09.551521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-13T17:43:56.017363+0000","last_change":"2024-09-13T17:43:36.972254+0000","last_active":"2024-09-13T17:43:56.017363+0000","last_peered":"2024-09-13T17:43:56.017363+0000","last_clean":"2024-09-13T17:43:56.017363+0000","last_became_active":"2024-09-13T17:43:36.971289+0000","last_became_peered":"2024-09-13T17:43:36.971289+0000","last_unstale":"2024-09-13T17:43:56.017363+0000","last_undegraded":"2024-09-13T17:43:56.017363+0000","last_fullsized":"2024-09-13T17:43:56.017363+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:43:36.560926+0000","last_clean_scrub_stamp":"2024-09-13T17:43:36.560926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:02:09.542526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":3},{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":4994,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":50,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":44,"ondisk_log_size":44,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":394,"num_read_kb":709,"num_write":197,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738395,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":155325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.498}]},{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738396,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32092,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":995328,"data_stored":734655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.443},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.46200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48699999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738395,"num_pgs":137,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":1011712,"data_stored":740859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29899999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.29699999999999999}]},{"osd":2,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.33700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":4924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:00.342 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:44:00.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:44:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.484+0000 7fe75720e640 1 Processor -- start 2024-09-13T17:44:00.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.484+0000 7fe75720e640 1 -- start start 2024-09-13T17:44:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe750134e90 0x7fe750071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750070de0 con 0x7fe750135270 2024-09-13T17:44:00.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe750071d60 con 0x7fe750134e90 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe754faa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe750134e90 0x7fe750071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe747fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe750135270 0x7fe75006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57736/0 (socket says 172.21.15.96:57736) 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe754faa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe750134e90 0x7fe750071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49034/0 (socket says 172.21.15.96:49034) 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe747fff640 1 -- 172.21.15.96:0/4192926252 learned_addr learned my addr 172.21.15.96:0/4192926252 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe754faa640 1 -- 172.21.15.96:0/4192926252 >> v1:172.21.15.96:6789/0 conn(0x7fe750135270 legacy=0x7fe75006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe7477fe640 1 -- 172.21.15.96:0/4192926252 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032163706 0 0) 0x7fe750070de0 con 0x7fe750135270 2024-09-13T17:44:00.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe754faa640 1 -- 172.21.15.96:0/4192926252 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75006f110 con 0x7fe750134e90 2024-09-13T17:44:00.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe754faa640 1 --2- 172.21.15.96:0/4192926252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe750071530 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009740 tx=0x7fe74c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47599f43fc692571 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe7477fe640 1 -- 172.21.15.96:0/4192926252 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0ab530 con 0x7fe750134e90 2024-09-13T17:44:00.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe7477fe640 1 -- 172.21.15.96:0/4192926252 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c0a7050 con 0x7fe750134e90 2024-09-13T17:44:00.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe7477fe640 1 -- 172.21.15.96:0/4192926252 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0ab7e0 con 0x7fe750134e90 2024-09-13T17:44:00.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- 172.21.15.96:0/4192926252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 msgr2=0x7fe750071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 --2- 172.21.15.96:0/4192926252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe750071530 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009740 tx=0x7fe74c01c390 comp rx=0 tx=0).stop 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- 172.21.15.96:0/4192926252 shutdown_connections 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 --2- 172.21.15.96:0/4192926252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe750071530 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- 172.21.15.96:0/4192926252 >> 172.21.15.96:0/4192926252 conn(0x7fe750098a60 msgr2=0x7fe75009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.488+0000 7fe75720e640 1 -- 172.21.15.96:0/4192926252 shutdown_connections 2024-09-13T17:44:00.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- 172.21.15.96:0/4192926252 wait complete. 2024-09-13T17:44:00.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 Processor -- start 2024-09-13T17:44:00.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- start start 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750133960 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe754faa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe754faa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49048/0 (socket says 172.21.15.96:49048) 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe754faa640 1 -- 172.21.15.96:0/3302036376 learned_addr learned my addr 172.21.15.96:0/3302036376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe754faa640 1 -- 172.21.15.96:0/3302036376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c009420 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe754faa640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe74c019fc0 tx=0x7fe74c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0abea0 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c004020 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe74c0042c0 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7500a6010 con 0x7fe750134e90 2024-09-13T17:44:00.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7500a55d0 con 0x7fe750134e90 2024-09-13T17:44:00.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75011d2a0 con 0x7fe750134e90 2024-09-13T17:44:00.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fe74c0b2050 con 0x7fe750134e90 2024-09-13T17:44:00.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 0x7fe73c037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:00.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fe74c0e2200 con 0x7fe750134e90 2024-09-13T17:44:00.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe747fff640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 0x7fe73c037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:00.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.492+0000 7fe747fff640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 0x7fe73c037a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740007980 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:00.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.496+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe74c0b2300 con 0x7fe750134e90 2024-09-13T17:44:00.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe75006a1b0 con 0x7fe750134e90 2024-09-13T17:44:00.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe7457fa640 1 -- 172.21.15.96:0/3302036376 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+152 (secure 0 0 0) 0x7fe74c0b0070 con 0x7fe750134e90 2024-09-13T17:44:00.684 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:44:00.684 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs_b, metadata pool: cephfs_b_metadata, data pools: [cephfs_b_data ] 2024-09-13T17:44:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 msgr2=0x7fe73c037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 0x7fe73c037a30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe740007fd0 tx=0x7fe740007980 comp rx=0 tx=0).stop 2024-09-13T17:44:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 msgr2=0x7fe7501cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe74c019fc0 tx=0x7fe74c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 shutdown_connections 2024-09-13T17:44:00.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe73c035570 0x7fe73c037a30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 --2- 172.21.15.96:0/3302036376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe750134e90 0x7fe7501cc8c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:00.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 >> 172.21.15.96:0/3302036376 conn(0x7fe750098a60 msgr2=0x7fe750133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:00.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 shutdown_connections 2024-09-13T17:44:00.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:00.680+0000 7fe75720e640 1 -- 172.21.15.96:0/3302036376 wait complete. 2024-09-13T17:44:00.701 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs_b; just confirmed its presence on cluster 2024-09-13T17:44:00.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:44:00.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:44:00.701 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:44:00.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:00.702 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs_b 2024-09-13T17:44:00.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:44:00.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:44:00.702 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:00.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.702 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:44:00.709 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:44:00.709 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:44:00.710 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42442sec preferred_lft 42442sec 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:00.711 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:44:00.712 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:44:00.713 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:00.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.713 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:44:00.760 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:44:00.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.762 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:44:00.811 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:44:00.811 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:44:00.811 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:44:00.811 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:44:00.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.812 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:44:00.812 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:44:00.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:44:00.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:44:00.813 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:44:00.813 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:44:00.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.886 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:44:00.931 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:44:00.931 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:00.931 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:00.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:00.931 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:44:00.984 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:44:00.985 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:00.985 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:01.032 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:01.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:01.032 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:01.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:01.136 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:01.136 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:01.189 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:01.190 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:01.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:01.191 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:01.239 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:01.240 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:44:01.240 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_fs=cephfs_b) 2024-09-13T17:44:01.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:01.300 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:01.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:01.353 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:01.354 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:01.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:01.355 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:01.368 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:44:01.364+0000 7f6fa87a6580 -1 init, newargv = 0x562f9109dca0 newargc=15 2024-09-13T17:44:01.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[39400]: starting ceph client 2024-09-13T17:44:01.373 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:01.373 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[39400]: starting fuse 2024-09-13T17:44:02.474 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:02.492 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:02.492 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:02.546 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:02.548 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:02.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:02.548 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:02.596 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:02.596 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:44:02.596 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:44:02.596 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:02.596 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:02.596 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:02.597 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:02.598 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:02.716 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:02.899 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:02.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:02.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok status 2024-09-13T17:44:03.059 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:03.059 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:44:03.059 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "39400", 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:03.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.061 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 192.168.0.2:0/1234192158", 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/1234192158", 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 25, 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 42, 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs_b" 2024-09-13T17:44:03.062 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:03.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:03.073 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:44:03.073 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:03.079 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:44:03.079 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:03.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:03.080 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:03.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:03.136 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dumpfs=cephfs_b) 2024-09-13T17:44:03.262 INFO:teuthology.orchestra.run.smithi096.stdout:{"cephfs_b": {}} 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:03.270 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:03.271 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:03.272 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:03.272 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:03.332 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:03.518 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:03.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:03.519 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok mds_sessions 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:03.678 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:44:03.679 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:03.680 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:45:02.370658+0000", 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:44:02.370658+0000", 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25 2024-09-13T17:44:03.681 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:04.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:04.692 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dumpfs=cephfs_b) 2024-09-13T17:44:04.781 INFO:teuthology.orchestra.run.smithi096.stdout:{"cephfs_b": {}} 2024-09-13T17:44:04.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:04.790 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:04.791 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:04.792 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:04.792 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:04.792 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:04.851 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:05.039 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:05.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:05.040 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok mds_sessions 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:05.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:44:05.198 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:44:05.199 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:45:02.370658+0000", 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:44:02.370658+0000", 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25 2024-09-13T17:44:05.200 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:06.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:06.212 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dumpfs=cephfs_b) 2024-09-13T17:44:06.298 INFO:teuthology.orchestra.run.smithi096.stdout:{"cephfs_b": {}} 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:06.307 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:06.308 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:06.309 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:06.368 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:06.555 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:06.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:06.556 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok mds_sessions 2024-09-13T17:44:06.709 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:06.710 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:06.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:45:02.370658+0000", 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:44:02.370658+0000", 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:44:06.712 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:44:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:44:06.713 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25 2024-09-13T17:44:06.713 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:07.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:07.724 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dumpfs=cephfs_b) 2024-09-13T17:44:07.810 INFO:teuthology.orchestra.run.smithi096.stdout:{"cephfs_b": {}} 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:07.819 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:07.820 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:07.821 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:07.821 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:07.821 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:07.884 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:08.070 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:08.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:08.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok mds_sessions 2024-09-13T17:44:08.227 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:08.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:08.228 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:08.229 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:08.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:45:02.370658+0000", 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:44:02.370658+0000", 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25 2024-09-13T17:44:08.231 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:09.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:09.243 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --dumpfs=cephfs_b) 2024-09-13T17:44:09.329 INFO:teuthology.orchestra.run.smithi096.stdout:{"cephfs_b": {"4917": {"mount_root": "/", "chit": 66.67, "dlease": 0.0, "ofiles": "0", "oicaps": "1", "oinodes": "0", "rtio": 0.0, "raio": 0.0, "rsp": 0.0, "wtio": 0.0, "waio": 0.0, "wsp": 0.0, "rlatavg": 0.0, "rlatsd": 0.0, "wlatavg": 0.0, "wlatsd": 0.0, "mlatavg": 0.73, "mlatsd": 0.35, "mount_point@host/addr": "/home/ubuntu/cephtest/mnt.1@smithi096/192.168.0.2"}}} 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:09.338 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:09.339 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:09.399 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:09.587 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.39400.asok 2024-09-13T17:44:09.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:09.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.39400.asok mds_sessions 2024-09-13T17:44:09.742 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4917, 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4917 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1234192158 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4917 v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "v1:192.168.0.2:0/1234192158", 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions": [ 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": 0, 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "addrs": { 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6826", 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:09.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6827", 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 285199718 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "seq": 0, 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_gen": 0, 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_ttl": "2024-09-13T17:45:02.370658+0000", 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "last_cap_renew_request": "2024-09-13T17:44:02.370658+0000", 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_renew_seq": 2, 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "num_caps": 1, 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "open" 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:09.745 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-09-13T17:44:09.746 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25 2024-09-13T17:44:09.746 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:09.756 DEBUG:tasks.cephfs.test_fstop:metrics={'cephfs_b': {'4917': {'mount_root': '/', 'chit': 66.67, 'dlease': 0.0, 'ofiles': '0', 'oicaps': '1', 'oinodes': '0', 'rtio': 0.0, 'raio': 0.0, 'rsp': 0.0, 'wtio': 0.0, 'waio': 0.0, 'wsp': 0.0, 'rlatavg': 0.0, 'rlatsd': 0.0, 'wlatavg': 0.0, 'wlatsd': 0.0, 'mlatavg': 0.73, 'mlatsd': 0.35, 'mount_point@host/addr': '/home/ubuntu/cephtest/mnt.1@smithi096/192.168.0.2'}}} 2024-09-13T17:44:09.756 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:09.756 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:09.762 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:09.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e42861640 1 Processor -- start 2024-09-13T17:44:09.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e42861640 1 -- start start 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e42861640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e42861640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c06e130 con 0x7f5e3c09dc20 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e42861640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0a6750 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e3bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.920+0000 7f5e3b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5e3c09dc20 0x7f5e3c134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40040/0 (socket says 172.21.15.96:40040) 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34548/0 (socket says 172.21.15.96:34548) 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3b7fe640 1 -- 172.21.15.96:0/1571071138 learned_addr learned my addr 172.21.15.96:0/1571071138 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3bfff640 1 -- 172.21.15.96:0/1571071138 >> v1:172.21.15.96:6789/0 conn(0x7f5e3c09dc20 legacy=0x7f5e3c134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3bfff640 1 -- 172.21.15.96:0/1571071138 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0712e0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3affd640 1 -- 172.21.15.96:0/1571071138 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436315747 0 0) 0x7f5e3c06e130 con 0x7f5e3c09dc20 2024-09-13T17:44:09.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3bfff640 1 --2- 172.21.15.96:0/1571071138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009740 tx=0x7f5e2c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bc861ef921ebfd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3affd640 1 -- 172.21.15.96:0/1571071138 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2c0ab400 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3affd640 1 -- 172.21.15.96:0/1571071138 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0a7050 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e3affd640 1 -- 172.21.15.96:0/1571071138 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2c0ab6b0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 -- 172.21.15.96:0/1571071138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 msgr2=0x7f5e3c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 --2- 172.21.15.96:0/1571071138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c009740 tx=0x7f5e2c01c390 comp rx=0 tx=0).stop 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 -- 172.21.15.96:0/1571071138 shutdown_connections 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 --2- 172.21.15.96:0/1571071138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c134ab0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 -- 172.21.15.96:0/1571071138 >> 172.21.15.96:0/1571071138 conn(0x7f5e3c098a60 msgr2=0x7f5e3c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 -- 172.21.15.96:0/1571071138 shutdown_connections 2024-09-13T17:44:09.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.924+0000 7f5e42861640 1 -- 172.21.15.96:0/1571071138 wait complete. 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 Processor -- start 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 -- start start 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0a55b0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34558/0 (socket says 172.21.15.96:34558) 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3bfff640 1 -- 172.21.15.96:0/4050531222 learned_addr learned my addr 172.21.15.96:0/4050531222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:09.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3bfff640 1 -- 172.21.15.96:0/4050531222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009420 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3bfff640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c019fc0 tx=0x7f5e2c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2c0b7450 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c0b75b0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e2c0b7850 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0a3810 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0a42a0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c11c930 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f5e2c0b79b0 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 0x7f5e28037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f5e2c0e2850 con 0x7f5e3c06f6d0 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3b7fe640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 0x7f5e28037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:09.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.928+0000 7f5e3b7fe640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 0x7f5e28037a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5e30007fd0 tx=0x7f5e30007920 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:09.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:09.932+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e2c0c7b80 con 0x7f5e3c06f6d0 2024-09-13T17:44:10.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.092+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e3c134ab0 con 0x7f5e3c06f6d0 2024-09-13T17:44:10.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.096+0000 7f5e38ff9640 1 -- 172.21.15.96:0/4050531222 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14765 (secure 0 0 0) 0x7f5e2c0bb020 con 0x7f5e3c06f6d0 2024-09-13T17:44:10.101 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:10.101 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:57.009683+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:10.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 msgr2=0x7f5e28037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 0x7f5e28037a80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5e30007fd0 tx=0x7f5e30007920 comp rx=0 tx=0).stop 2024-09-13T17:44:10.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 msgr2=0x7f5e3c0a25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c019fc0 tx=0x7f5e2c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 shutdown_connections 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f5e280355c0 0x7f5e28037a80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 --2- 172.21.15.96:0/4050531222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e3c06f6d0 0x7f5e3c0a25b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 >> 172.21.15.96:0/4050531222 conn(0x7f5e3c098a60 msgr2=0x7f5e3c09ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 shutdown_connections 2024-09-13T17:44:10.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.100+0000 7f5e42861640 1 -- 172.21.15.96:0/4050531222 wait complete. 2024-09-13T17:44:10.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:10.122 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:10.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66cc45a640 1 Processor -- start 2024-09-13T17:44:10.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66cc45a640 1 -- start start 2024-09-13T17:44:10.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66cc45a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66c406f6d0 0x7f66c4134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66cc45a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c406e130 con 0x7f66c409dc20 2024-09-13T17:44:10.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66cc45a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c40a6750 con 0x7f66c406f6d0 2024-09-13T17:44:10.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66ca1f6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66c406f6d0 0x7f66c4134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c99f5640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f66c409dc20 0x7f66c4134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40054/0 (socket says 172.21.15.96:40054) 2024-09-13T17:44:10.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c99f5640 1 -- 172.21.15.96:0/3912356398 learned_addr learned my addr 172.21.15.96:0/3912356398 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:10.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66ca1f6640 1 -- 172.21.15.96:0/3912356398 >> v1:172.21.15.96:6789/0 conn(0x7f66c409dc20 legacy=0x7f66c4134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66ca1f6640 1 -- 172.21.15.96:0/3912356398 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c40712e0 con 0x7f66c406f6d0 2024-09-13T17:44:10.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c91f4640 1 -- 172.21.15.96:0/3912356398 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904642834 0 0) 0x7f66c406e130 con 0x7f66c409dc20 2024-09-13T17:44:10.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66ca1f6640 1 --2- 172.21.15.96:0/3912356398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c4134ab0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009740 tx=0x7f66c001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f060ae44f52377a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c91f4640 1 -- 172.21.15.96:0/3912356398 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00ab400 con 0x7f66c406f6d0 2024-09-13T17:44:10.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c91f4640 1 -- 172.21.15.96:0/3912356398 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c00a7050 con 0x7f66c406f6d0 2024-09-13T17:44:10.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.244+0000 7f66c91f4640 1 -- 172.21.15.96:0/3912356398 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00ab6b0 con 0x7f66c406f6d0 2024-09-13T17:44:10.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/3912356398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 msgr2=0x7f66c4134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 --2- 172.21.15.96:0/3912356398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c4134ab0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f66c0009740 tx=0x7f66c001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:10.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/3912356398 shutdown_connections 2024-09-13T17:44:10.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 --2- 172.21.15.96:0/3912356398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c4134ab0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/3912356398 >> 172.21.15.96:0/3912356398 conn(0x7f66c4098a60 msgr2=0x7f66c409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/3912356398 shutdown_connections 2024-09-13T17:44:10.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/3912356398 wait complete. 2024-09-13T17:44:10.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 Processor -- start 2024-09-13T17:44:10.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- start start 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c40a3ff0 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66ca1f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66ca1f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34570/0 (socket says 172.21.15.96:34570) 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66ca1f6640 1 -- 172.21.15.96:0/2831478286 learned_addr learned my addr 172.21.15.96:0/2831478286 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66ca1f6640 1 -- 172.21.15.96:0/2831478286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c0009420 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66ca1f6640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f66c0019fc0 tx=0x7f66c001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00b7450 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c00b75b0 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c40a5200 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66c00b7850 con 0x7f66c406f6d0 2024-09-13T17:44:10.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c40a3630 con 0x7f66c406f6d0 2024-09-13T17:44:10.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c40a61f0 con 0x7f66c406f6d0 2024-09-13T17:44:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f66c00b79b0 con 0x7f66c406f6d0 2024-09-13T17:44:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 0x7f66a8037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f66c00e2980 con 0x7f66c406f6d0 2024-09-13T17:44:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.248+0000 7f66c99f5640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 0x7f66a8037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.252+0000 7f66c99f5640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 0x7f66a8037a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66b4007fd0 tx=0x7f66b4007920 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.256+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66c00c76c0 con 0x7f66c406f6d0 2024-09-13T17:44:10.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66c4138f80 con 0x7f66c406f6d0 2024-09-13T17:44:10.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66baffd640 1 -- 172.21.15.96:0/2831478286 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66c00b0070 con 0x7f66c406f6d0 2024-09-13T17:44:10.416 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:10.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 msgr2=0x7f66a8037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66cc45a640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 0x7f66a8037a80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66b4007fd0 tx=0x7f66b4007920 comp rx=0 tx=0).stop 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 msgr2=0x7f66c40a17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.412+0000 7f66cc45a640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f66c0019fc0 tx=0x7f66c001abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 shutdown_connections 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f66a80355c0 0x7f66a8037a80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 --2- 172.21.15.96:0/2831478286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66c406f6d0 0x7f66c40a17c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 >> 172.21.15.96:0/2831478286 conn(0x7f66c4098a60 msgr2=0x7f66c4134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 shutdown_connections 2024-09-13T17:44:10.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.416+0000 7f66cc45a640 1 -- 172.21.15.96:0/2831478286 wait complete. 2024-09-13T17:44:10.436 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:10.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.552+0000 7f0be50b9640 1 Processor -- start 2024-09-13T17:44:10.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0be50b9640 1 -- start start 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0be50b9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0be013af50 0x7f0be013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0be50b9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be0138050 con 0x7f0be009d690 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0be50b9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be009c410 con 0x7f0be013af50 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bded9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0be009d690 0x7f0be0138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40066/0 (socket says 172.21.15.96:40066) 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bde59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0be013af50 0x7f0be013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bded9d640 1 -- 172.21.15.96:0/3691704237 learned_addr learned my addr 172.21.15.96:0/3691704237 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bde59c640 1 -- 172.21.15.96:0/3691704237 >> v1:172.21.15.96:6789/0 conn(0x7f0be009d690 legacy=0x7f0be0138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bde59c640 1 -- 172.21.15.96:0/3691704237 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be009cba0 con 0x7f0be013af50 2024-09-13T17:44:10.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bde59c640 1 --2- 172.21.15.96:0/3691704237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be013af50 0x7f0be013d340 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009740 tx=0x7f0bd401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a5975ad2531447 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bddd9b640 1 -- 172.21.15.96:0/3691704237 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd40a8840 con 0x7f0be013af50 2024-09-13T17:44:10.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bddd9b640 1 -- 172.21.15.96:0/3691704237 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd40a89a0 con 0x7f0be013af50 2024-09-13T17:44:10.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.556+0000 7f0bddd9b640 1 -- 172.21.15.96:0/3691704237 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd40a8c60 con 0x7f0be013af50 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/3691704237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be013af50 msgr2=0x7f0be013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 --2- 172.21.15.96:0/3691704237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be013af50 0x7f0be013d340 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4009740 tx=0x7f0bd401ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/3691704237 shutdown_connections 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 --2- 172.21.15.96:0/3691704237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be013af50 0x7f0be013d340 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/3691704237 >> 172.21.15.96:0/3691704237 conn(0x7f0be006bc60 msgr2=0x7f0be006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/3691704237 shutdown_connections 2024-09-13T17:44:10.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/3691704237 wait complete. 2024-09-13T17:44:10.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 Processor -- start 2024-09-13T17:44:10.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- start start 2024-09-13T17:44:10.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be00a6f60 con 0x7f0be009d690 2024-09-13T17:44:10.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bded9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bded9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34576/0 (socket says 172.21.15.96:34576) 2024-09-13T17:44:10.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bded9d640 1 -- 172.21.15.96:0/4157407364 learned_addr learned my addr 172.21.15.96:0/4157407364 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bded9d640 1 -- 172.21.15.96:0/4157407364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4009420 con 0x7f0be009d690 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bded9d640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c2f0 tx=0x7f0bd000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd0005c20 con 0x7f0be009d690 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bd0005d80 con 0x7f0be009d690 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bd0007200 con 0x7f0be009d690 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be00a5fe0 con 0x7f0be009d690 2024-09-13T17:44:10.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.560+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be00a4d60 con 0x7f0be009d690 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f0bd0007360 con 0x7f0be009d690 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be009da70 con 0x7f0be009d690 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0bc77fe640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 0x7f0bcc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f0bd0047020 con 0x7f0be009d690 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0bde59c640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 0x7f0bcc037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.564+0000 7f0bde59c640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 0x7f0bcc037a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40b2000 tx=0x7f0bd40aa070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.568+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0bd00198b0 con 0x7f0be009d690 2024-09-13T17:44:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0be0138050 con 0x7f0be009d690 2024-09-13T17:44:10.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0bc77fe640 1 -- 172.21.15.96:0/4157407364 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0be0138050 con 0x7f0be009d690 2024-09-13T17:44:10.732 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 msgr2=0x7f0bcc037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0be50b9640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 0x7f0bcc037a30 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0bd40b2000 tx=0x7f0bd40aa070 comp rx=0 tx=0).stop 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 msgr2=0x7f0be00a6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.728+0000 7f0be50b9640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000c2f0 tx=0x7f0bd000c7b0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 shutdown_connections 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f0bcc035570 0x7f0bcc037a30 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 --2- 172.21.15.96:0/4157407364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0be009d690 0x7f0be00a6a20 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 >> 172.21.15.96:0/4157407364 conn(0x7f0be006bc60 msgr2=0x7f0be0139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 shutdown_connections 2024-09-13T17:44:10.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.732+0000 7f0be50b9640 1 -- 172.21.15.96:0/4157407364 wait complete. 2024-09-13T17:44:10.751 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:10.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1fff2640 1 Processor -- start 2024-09-13T17:44:10.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1fff2640 1 -- start start 2024-09-13T17:44:10.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1fff2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1b1813af50 0x7f1b1813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1fff2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b18138050 con 0x7f1b1809d690 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1fff2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1809c410 con 0x7f1b1813af50 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1d58d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1b1813af50 0x7f1b1813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1dd8e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1b1809d690 0x7f1b18138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40080/0 (socket says 172.21.15.96:40080) 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.864+0000 7f1b1dd8e640 1 -- 172.21.15.96:0/1750272131 learned_addr learned my addr 172.21.15.96:0/1750272131 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1d58d640 1 -- 172.21.15.96:0/1750272131 >> v1:172.21.15.96:6789/0 conn(0x7f1b1809d690 legacy=0x7f1b18138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1d58d640 1 -- 172.21.15.96:0/1750272131 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b1809cba0 con 0x7f1b1813af50 2024-09-13T17:44:10.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1d58d640 1 --2- 172.21.15.96:0/1750272131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1813af50 0x7f1b1813d340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009740 tx=0x7f1b0c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eca7f1b85ce264 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1cd8c640 1 -- 172.21.15.96:0/1750272131 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b0c0a8840 con 0x7f1b1813af50 2024-09-13T17:44:10.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1cd8c640 1 -- 172.21.15.96:0/1750272131 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0c0a89a0 con 0x7f1b1813af50 2024-09-13T17:44:10.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1cd8c640 1 -- 172.21.15.96:0/1750272131 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b0c0a8c60 con 0x7f1b1813af50 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- 172.21.15.96:0/1750272131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1813af50 msgr2=0x7f1b1813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/1750272131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1813af50 0x7f1b1813d340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c009740 tx=0x7f1b0c01ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- 172.21.15.96:0/1750272131 shutdown_connections 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/1750272131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1813af50 0x7f1b1813d340 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- 172.21.15.96:0/1750272131 >> 172.21.15.96:0/1750272131 conn(0x7f1b1806bc60 msgr2=0x7f1b1806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- 172.21.15.96:0/1750272131 shutdown_connections 2024-09-13T17:44:10.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- 172.21.15.96:0/1750272131 wait complete. 2024-09-13T17:44:10.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 Processor -- start 2024-09-13T17:44:10.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.868+0000 7f1b1fff2640 1 -- start start 2024-09-13T17:44:10.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1fff2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1fff2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180a4f90 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1dd8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1dd8e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34590/0 (socket says 172.21.15.96:34590) 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1dd8e640 1 -- 172.21.15.96:0/395148061 learned_addr learned my addr 172.21.15.96:0/395148061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1dd8e640 1 -- 172.21.15.96:0/395148061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c009420 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1dd8e640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cb40 tx=0x7f1b0800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b0800ab30 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800ac90 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b180a2c20 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b08012400 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b181373b0 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f1b08007070 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b180a3f70 con 0x7f1b1809d690 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 0x7f1b04037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:10.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f1b080478b0 con 0x7f1b1809d690 2024-09-13T17:44:10.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1d58d640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 0x7f1b04037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:10.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.872+0000 7f1b1d58d640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 0x7f1b04037a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b2000 tx=0x7f1b0c0aa070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:10.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:10.876+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1b180a3f70 con 0x7f1b1809d690 2024-09-13T17:44:11.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.032+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b18138050 con 0x7f1b1809d690 2024-09-13T17:44:11.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b167fc640 1 -- 172.21.15.96:0/395148061 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1b18138050 con 0x7f1b1809d690 2024-09-13T17:44:11.039 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:11.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 msgr2=0x7f1b04037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 0x7f1b04037a80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0b2000 tx=0x7f1b0c0aa070 comp rx=0 tx=0).stop 2024-09-13T17:44:11.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 msgr2=0x7f1b181c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cb40 tx=0x7f1b0800cf40 comp rx=0 tx=0).stop 2024-09-13T17:44:11.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 shutdown_connections 2024-09-13T17:44:11.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1b040355c0 0x7f1b04037a80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 --2- 172.21.15.96:0/395148061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b1809d690 0x7f1b181c9820 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 >> 172.21.15.96:0/395148061 conn(0x7f1b1806bc60 msgr2=0x7f1b18131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 shutdown_connections 2024-09-13T17:44:11.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.036+0000 7f1b1fff2640 1 -- 172.21.15.96:0/395148061 wait complete. 2024-09-13T17:44:11.056 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:11.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46f72c640 1 Processor -- start 2024-09-13T17:44:11.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46f72c640 1 -- start start 2024-09-13T17:44:11.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46f72c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa46809d690 0x7fa468138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46f72c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa468138050 con 0x7fa4681387d0 2024-09-13T17:44:11.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46f72c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46809c410 con 0x7fa46809d690 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46d4c8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa46809d690 0x7fa468138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46ccc7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa4681387d0 0x7fa468134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40086/0 (socket says 172.21.15.96:40086) 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46ccc7640 1 -- 172.21.15.96:0/1920099027 learned_addr learned my addr 172.21.15.96:0/1920099027 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46d4c8640 1 -- 172.21.15.96:0/1920099027 >> v1:172.21.15.96:6789/0 conn(0x7fa4681387d0 legacy=0x7fa468134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46d4c8640 1 -- 172.21.15.96:0/1920099027 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46809cba0 con 0x7fa46809d690 2024-09-13T17:44:11.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa45ffff640 1 -- 172.21.15.96:0/1920099027 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2710051082 0 0) 0x7fa464001120 con 0x7fa4681387d0 2024-09-13T17:44:11.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa46d4c8640 1 --2- 172.21.15.96:0/1920099027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa468138290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa458009740 tx=0x7fa45801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3787c3d1067349ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa45ffff640 1 -- 172.21.15.96:0/1920099027 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4580a8840 con 0x7fa46809d690 2024-09-13T17:44:11.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa45ffff640 1 -- 172.21.15.96:0/1920099027 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580a89a0 con 0x7fa46809d690 2024-09-13T17:44:11.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.168+0000 7fa45ffff640 1 -- 172.21.15.96:0/1920099027 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4580a8c60 con 0x7fa46809d690 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/1920099027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 msgr2=0x7fa468138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 --2- 172.21.15.96:0/1920099027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa468138290 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa458009740 tx=0x7fa45801ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/1920099027 shutdown_connections 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 --2- 172.21.15.96:0/1920099027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa468138290 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/1920099027 >> 172.21.15.96:0/1920099027 conn(0x7fa46806bc60 msgr2=0x7fa46806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/1920099027 shutdown_connections 2024-09-13T17:44:11.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/1920099027 wait complete. 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 Processor -- start 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- start start 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680a4c30 con 0x7fa46809d690 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46d4c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46d4c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34602/0 (socket says 172.21.15.96:34602) 2024-09-13T17:44:11.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46d4c8640 1 -- 172.21.15.96:0/2404828678 learned_addr learned my addr 172.21.15.96:0/2404828678 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46d4c8640 1 -- 172.21.15.96:0/2404828678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa458009420 con 0x7fa46809d690 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46d4c8640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa45801c840 tx=0x7fa45801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4580b0840 con 0x7fa46809d690 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4580b09a0 con 0x7fa46809d690 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4680a4e60 con 0x7fa46809d690 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa4580b0c60 con 0x7fa46809d690 2024-09-13T17:44:11.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4680a6ea0 con 0x7fa46809d690 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fa4580b7040 con 0x7fa46809d690 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46809bc00 con 0x7fa46809d690 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 0x7fa44c037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fa4580e5ba0 con 0x7fa46809d690 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.172+0000 7fa46ccc7640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 0x7fa44c037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.176+0000 7fa46ccc7640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 0x7fa44c037a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa464005890 tx=0x7fa464001280 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.180+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa46809bc00 con 0x7fa46809d690 2024-09-13T17:44:11.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.332+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa46809bc00 con 0x7fa46809d690 2024-09-13T17:44:11.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.332+0000 7fa45dffb640 1 -- 172.21.15.96:0/2404828678 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v42) v1 ==== 106+0+11 (secure 0 0 0) 0x7fa46809bc00 con 0x7fa46809d690 2024-09-13T17:44:11.338 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 msgr2=0x7fa44c037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 0x7fa44c037a80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa464005890 tx=0x7fa464001280 comp rx=0 tx=0).stop 2024-09-13T17:44:11.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 msgr2=0x7fa4680a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa45801c840 tx=0x7fa45801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 shutdown_connections 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fa44c0355c0 0x7fa44c037a80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 --2- 172.21.15.96:0/2404828678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa46809d690 0x7fa4680a6960 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 >> 172.21.15.96:0/2404828678 conn(0x7fa46806bc60 msgr2=0x7fa46809e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 shutdown_connections 2024-09-13T17:44:11.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.336+0000 7fa46f72c640 1 -- 172.21.15.96:0/2404828678 wait complete. 2024-09-13T17:44:11.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:11.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe90e761640 1 Processor -- start 2024-09-13T17:44:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe90e761640 1 -- start start 2024-09-13T17:44:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe90e761640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe90809dc20 0x7fe9081351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe90e761640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90806e130 con 0x7fe90806f6d0 2024-09-13T17:44:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe90e761640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9080a6750 con 0x7fe90809dc20 2024-09-13T17:44:11.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe9077fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe90809dc20 0x7fe9081351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe907fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe90806f6d0 0x7fe908134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40088/0 (socket says 172.21.15.96:40088) 2024-09-13T17:44:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe907fff640 1 -- 172.21.15.96:0/282939092 learned_addr learned my addr 172.21.15.96:0/282939092 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe9077fe640 1 -- 172.21.15.96:0/282939092 >> v1:172.21.15.96:6789/0 conn(0x7fe90806f6d0 legacy=0x7fe908134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe906ffd640 1 -- 172.21.15.96:0/282939092 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296082316 0 0) 0x7fe90806e130 con 0x7fe90806f6d0 2024-09-13T17:44:11.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe9077fe640 1 -- 172.21.15.96:0/282939092 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9080712e0 con 0x7fe90809dc20 2024-09-13T17:44:11.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe9077fe640 1 --2- 172.21.15.96:0/282939092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90809dc20 0x7fe9081351c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009740 tx=0x7fe8fc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a828d0477e66da5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe906ffd640 1 -- 172.21.15.96:0/282939092 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8fc0ab400 con 0x7fe90809dc20 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe906ffd640 1 -- 172.21.15.96:0/282939092 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0a7050 con 0x7fe90809dc20 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.476+0000 7fe906ffd640 1 -- 172.21.15.96:0/282939092 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8fc0ab6b0 con 0x7fe90809dc20 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/282939092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90809dc20 msgr2=0x7fe9081351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 --2- 172.21.15.96:0/282939092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90809dc20 0x7fe9081351c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009740 tx=0x7fe8fc01c390 comp rx=0 tx=0).stop 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/282939092 shutdown_connections 2024-09-13T17:44:11.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 --2- 172.21.15.96:0/282939092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90809dc20 0x7fe9081351c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/282939092 >> 172.21.15.96:0/282939092 conn(0x7fe908098a60 msgr2=0x7fe90809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/282939092 shutdown_connections 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/282939092 wait complete. 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 Processor -- start 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- start start 2024-09-13T17:44:11.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90809a370 con 0x7fe90806f6d0 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe907fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe907fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34620/0 (socket says 172.21.15.96:34620) 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe907fff640 1 -- 172.21.15.96:0/416087065 learned_addr learned my addr 172.21.15.96:0/416087065 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe907fff640 1 -- 172.21.15.96:0/416087065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc009420 con 0x7fe90806f6d0 2024-09-13T17:44:11.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe907fff640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800c420 tx=0x7fe8f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8f80074d0 con 0x7fe90806f6d0 2024-09-13T17:44:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8007630 con 0x7fe90806f6d0 2024-09-13T17:44:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe8f8004960 con 0x7fe90806f6d0 2024-09-13T17:44:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90811d7f0 con 0x7fe90806f6d0 2024-09-13T17:44:11.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90809a6c0 con 0x7fe90806f6d0 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fe8f8004ac0 con 0x7fe90806f6d0 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe908134ab0 con 0x7fe90806f6d0 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 0x7fe8f4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fe8f8047370 con 0x7fe90806f6d0 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.480+0000 7fe9077fe640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 0x7fe8f4037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.484+0000 7fe9077fe640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 0x7fe8f4037a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ad000 tx=0x7fe8fc008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.488+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe8f801ab40 con 0x7fe90806f6d0 2024-09-13T17:44:11.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7fe908138f60 con 0x7fe90806f6d0 2024-09-13T17:44:11.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe904ff9640 1 -- 172.21.15.96:0/416087065 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v42) v1 ==== 102+0+11 (secure 0 0 0) 0x7fe8f8018070 con 0x7fe90806f6d0 2024-09-13T17:44:11.649 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:11.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 msgr2=0x7fe8f4037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 0x7fe8f4037a80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0ad000 tx=0x7fe8fc008040 comp rx=0 tx=0).stop 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 msgr2=0x7fe908098e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800c420 tx=0x7fe8f800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 shutdown_connections 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe8f40355c0 0x7fe8f4037a80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 --2- 172.21.15.96:0/416087065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90806f6d0 0x7fe908098e40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 >> 172.21.15.96:0/416087065 conn(0x7fe908098a60 msgr2=0x7fe908133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 shutdown_connections 2024-09-13T17:44:11.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.644+0000 7fe90e761640 1 -- 172.21.15.96:0/416087065 wait complete. 2024-09-13T17:44:11.668 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:44:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8fac76640 1 Processor -- start 2024-09-13T17:44:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8fac76640 1 -- start start 2024-09-13T17:44:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8fac76640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc8f4134e70 0x7fc8f409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8fac76640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f409ed60 con 0x7fc8f4135250 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8fac76640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f409ddc0 con 0x7fc8f4134e70 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f8a12640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc8f4134e70 0x7fc8f409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc8f4135250 0x7fc8f4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40092/0 (socket says 172.21.15.96:40092) 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f3fff640 1 -- 172.21.15.96:0/1066416742 learned_addr learned my addr 172.21.15.96:0/1066416742 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f8a12640 1 -- 172.21.15.96:0/1066416742 >> v1:172.21.15.96:6789/0 conn(0x7fc8f4135250 legacy=0x7fc8f4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f37fe640 1 -- 172.21.15.96:0/1066416742 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104316823 0 0) 0x7fc8e4001120 con 0x7fc8f4135250 2024-09-13T17:44:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.780+0000 7fc8f8a12640 1 -- 172.21.15.96:0/1066416742 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40a68b0 con 0x7fc8f4134e70 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8f8a12640 1 --2- 172.21.15.96:0/1066416742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f409e820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009740 tx=0x7fc8e801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6181087db6a65909 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8f37fe640 1 -- 172.21.15.96:0/1066416742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e80ab530 con 0x7fc8f4134e70 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8f37fe640 1 -- 172.21.15.96:0/1066416742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e80a7050 con 0x7fc8f4134e70 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8f37fe640 1 -- 172.21.15.96:0/1066416742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e80ab7e0 con 0x7fc8f4134e70 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- 172.21.15.96:0/1066416742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 msgr2=0x7fc8f409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 --2- 172.21.15.96:0/1066416742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f409e820 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009740 tx=0x7fc8e801c420 comp rx=0 tx=0).stop 2024-09-13T17:44:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- 172.21.15.96:0/1066416742 shutdown_connections 2024-09-13T17:44:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 --2- 172.21.15.96:0/1066416742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f409e820 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- 172.21.15.96:0/1066416742 >> 172.21.15.96:0/1066416742 conn(0x7fc8f4098a60 msgr2=0x7fc8f409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- 172.21.15.96:0/1066416742 shutdown_connections 2024-09-13T17:44:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- 172.21.15.96:0/1066416742 wait complete. 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 Processor -- start 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.784+0000 7fc8fac76640 1 -- start start 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8fac76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8fac76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40a2fe0 con 0x7fc8f4134e70 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f8a12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f8a12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34634/0 (socket says 172.21.15.96:34634) 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f8a12640 1 -- 172.21.15.96:0/2957158872 learned_addr learned my addr 172.21.15.96:0/2957158872 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:11.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f8a12640 1 -- 172.21.15.96:0/2957158872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e8009420 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f8a12640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8019030 tx=0x7fc8e801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e80abde0 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e8004030 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8e80042d0 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f40a0f20 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f40a1d50 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fc8e8003620 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f41368b0 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 0x7fc8e0037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fc8e80e2ab0 con 0x7fc8f4134e70 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f3fff640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 0x7fc8e0037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:11.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.788+0000 7fc8f3fff640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 0x7fc8e0037a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4000c10 tx=0x7fc8e4001320 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:11.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.792+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc8e80aa970 con 0x7fc8f4134e70 2024-09-13T17:44:11.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8f40a0440 con 0x7fc8f4134e70 2024-09-13T17:44:11.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8f17fa640 1 -- 172.21.15.96:0/2957158872 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v42) v1 ==== 80+0+467 (secure 0 0 0) 0x7fc8e80b0030 con 0x7fc8f4134e70 2024-09-13T17:44:11.968 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2785343960 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3263021005 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2962687900 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/715245426 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3193678716 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3614824494 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:11.969 INFO:teuthology.orchestra.run.smithi096.stderr:listed 8 entries 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 msgr2=0x7fc8e0037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 0x7fc8e0037a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4000c10 tx=0x7fc8e4001320 comp rx=0 tx=0).stop 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 msgr2=0x7fc8f40a27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8019030 tx=0x7fc8e801ac60 comp rx=0 tx=0).stop 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 shutdown_connections 2024-09-13T17:44:11.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fc8e00355c0 0x7fc8e0037a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 --2- 172.21.15.96:0/2957158872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8f4134e70 0x7fc8f40a27b0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 >> 172.21.15.96:0/2957158872 conn(0x7fc8f4098a60 msgr2=0x7fc8f4099380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 shutdown_connections 2024-09-13T17:44:11.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:11.964+0000 7fc8fac76640 1 -- 172.21.15.96:0/2957158872 wait complete. 2024-09-13T17:44:11.986 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:11.986 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:11.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c424cc640 1 Processor -- start 2024-09-13T17:44:12.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c424cc640 1 -- start start 2024-09-13T17:44:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c424cc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6c3c09e820 0x7f6c3c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c424cc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c3c0a68b0 con 0x7f6c3c134e70 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c424cc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3c09ddc0 con 0x7f6c3c09e820 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c3b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6c3c09e820 0x7f6c3c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c3bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6c3c134e70 0x7f6c3c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:40094/0 (socket says 172.21.15.96:40094) 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c3b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6c3c09e820 0x7f6c3c13ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34646/0 (socket says 172.21.15.96:34646) 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.140+0000 7f6c3bfff640 1 -- 172.21.15.96:0/1179349091 learned_addr learned my addr 172.21.15.96:0/1179349091 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3b7fe640 1 -- 172.21.15.96:0/1179349091 >> v1:172.21.15.96:6789/0 conn(0x7f6c3c134e70 legacy=0x7f6c3c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3b7fe640 1 -- 172.21.15.96:0/1179349091 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c136970 con 0x7f6c3c09e820 2024-09-13T17:44:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3b7fe640 1 --2- 172.21.15.96:0/1179349091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c13ae50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009d90 tx=0x7f6c2401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42948dbac378b24d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3affd640 1 -- 172.21.15.96:0/1179349091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c240a57b0 con 0x7f6c3c09e820 2024-09-13T17:44:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3affd640 1 -- 172.21.15.96:0/1179349091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c240a5910 con 0x7f6c3c09e820 2024-09-13T17:44:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c3affd640 1 -- 172.21.15.96:0/1179349091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c240a5bd0 con 0x7f6c3c09e820 2024-09-13T17:44:12.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 -- 172.21.15.96:0/1179349091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 msgr2=0x7f6c3c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 --2- 172.21.15.96:0/1179349091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c13ae50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009d90 tx=0x7f6c2401c970 comp rx=0 tx=0).stop 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 -- 172.21.15.96:0/1179349091 shutdown_connections 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 --2- 172.21.15.96:0/1179349091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c13ae50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 -- 172.21.15.96:0/1179349091 >> 172.21.15.96:0/1179349091 conn(0x7f6c3c098a60 msgr2=0x7f6c3c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 -- 172.21.15.96:0/1179349091 shutdown_connections 2024-09-13T17:44:12.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.144+0000 7f6c424cc640 1 -- 172.21.15.96:0/1179349091 wait complete. 2024-09-13T17:44:12.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 Processor -- start 2024-09-13T17:44:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 -- start start 2024-09-13T17:44:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3c0a3070 con 0x7f6c3c09e820 2024-09-13T17:44:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34660/0 (socket says 172.21.15.96:34660) 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3bfff640 1 -- 172.21.15.96:0/2806761237 learned_addr learned my addr 172.21.15.96:0/2806761237 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3bfff640 1 -- 172.21.15.96:0/2806761237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c24009a70 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3bfff640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c3a0 tx=0x7f6c3400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c34005c20 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34005d80 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c34007200 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3c0a2e20 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c3c0a17b0 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f6c34007360 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c0a31c0 con 0x7f6c3c09e820 2024-09-13T17:44:12.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 0x7f6c1c037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f6c340471e0 con 0x7f6c3c09e820 2024-09-13T17:44:12.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3b7fe640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 0x7f6c1c037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.148+0000 7f6c3b7fe640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 0x7f6c1c037a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c240af000 tx=0x7f6c240a7070 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.152+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6c34012400 con 0x7f6c3c09e820 2024-09-13T17:44:12.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.300+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c3c06a1b0 con 0x7f6c3c09e820 2024-09-13T17:44:12.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c38ff9640 1 -- 172.21.15.96:0/2806761237 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14765 (secure 0 0 0) 0x7f6c34017020 con 0x7f6c3c09e820 2024-09-13T17:44:12.307 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:12.307 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:57.009683+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 msgr2=0x7f6c1c037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 0x7f6c1c037a80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c240af000 tx=0x7f6c240a7070 comp rx=0 tx=0).stop 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 msgr2=0x7f6c3c1cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400c3a0 tx=0x7f6c3400c860 comp rx=0 tx=0).stop 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 shutdown_connections 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6c1c0355c0 0x7f6c1c037a80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 --2- 172.21.15.96:0/2806761237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c3c09e820 0x7f6c3c1cda00 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 >> 172.21.15.96:0/2806761237 conn(0x7f6c3c098a60 msgr2=0x7f6c3c134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 shutdown_connections 2024-09-13T17:44:12.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.304+0000 7f6c424cc640 1 -- 172.21.15.96:0/2806761237 wait complete. 2024-09-13T17:44:12.325 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:12.326 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.444+0000 7f1f8e325640 1 Processor -- start 2024-09-13T17:44:12.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.444+0000 7f1f8e325640 1 -- start start 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f1f8809d690 0x7f1f88138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f88138050 con 0x7f1f881387d0 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8809c410 con 0x7f1f8809d690 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f877fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f1f881387d0 0x7f1f88134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38768/0 (socket says 172.21.15.96:38768) 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f877fe640 1 -- 172.21.15.96:0/468857607 learned_addr learned my addr 172.21.15.96:0/468857607 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:12.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f87fff640 1 --2- 172.21.15.96:0/468857607 >> v2:172.21.15.96:3300/0 conn(0x7f1f8809d690 0x7f1f88138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f87fff640 1 -- 172.21.15.96:0/468857607 >> v1:172.21.15.96:6789/0 conn(0x7f1f881387d0 legacy=0x7f1f88134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f87fff640 1 -- 172.21.15.96:0/468857607 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8809cba0 con 0x7f1f8809d690 2024-09-13T17:44:12.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f87fff640 1 --2- 172.21.15.96:0/468857607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f88138290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009740 tx=0x7f1f7801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b711fbcc1f0ef5f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f86ffd640 1 -- 172.21.15.96:0/468857607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780a8840 con 0x7f1f8809d690 2024-09-13T17:44:12.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f86ffd640 1 -- 172.21.15.96:0/468857607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780a89a0 con 0x7f1f8809d690 2024-09-13T17:44:12.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f86ffd640 1 -- 172.21.15.96:0/468857607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780a8c60 con 0x7f1f8809d690 2024-09-13T17:44:12.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 -- 172.21.15.96:0/468857607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 msgr2=0x7f1f88138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 --2- 172.21.15.96:0/468857607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f88138290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009740 tx=0x7f1f7801ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 -- 172.21.15.96:0/468857607 shutdown_connections 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.448+0000 7f1f8e325640 1 --2- 172.21.15.96:0/468857607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f88138290 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/468857607 >> 172.21.15.96:0/468857607 conn(0x7f1f8806bc60 msgr2=0x7f1f8806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/468857607 shutdown_connections 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/468857607 wait complete. 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 Processor -- start 2024-09-13T17:44:12.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- start start 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880a5e10 con 0x7f1f8809d690 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55128/0 (socket says 172.21.15.96:55128) 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f87fff640 1 -- 172.21.15.96:0/1909475300 learned_addr learned my addr 172.21.15.96:0/1909475300 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:12.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f87fff640 1 -- 172.21.15.96:0/1909475300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009420 con 0x7f1f8809d690 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f87fff640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1f7801c840 tx=0x7f1f7801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780b0840 con 0x7f1f8809d690 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f780b09a0 con 0x7f1f8809d690 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f880a70a0 con 0x7f1f8809d690 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780b0c60 con 0x7f1f8809d690 2024-09-13T17:44:12.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f880a4300 con 0x7f1f8809d690 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8809bc00 con 0x7f1f8809d690 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f1f780b7040 con 0x7f1f8809d690 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 0x7f1f74037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f1f780e5b20 con 0x7f1f8809d690 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.452+0000 7f1f877fe640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 0x7f1f74037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.456+0000 7f1f877fe640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 0x7f1f74037a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005890 tx=0x7f1f7c007a90 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.460+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f1f8809bc00 con 0x7f1f8809d690 2024-09-13T17:44:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f8809bc00 con 0x7f1f8809d690 2024-09-13T17:44:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f84ff9640 1 -- 172.21.15.96:0/1909475300 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f8809bc00 con 0x7f1f8809d690 2024-09-13T17:44:12.624 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 msgr2=0x7f1f74037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 0x7f1f74037a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c005890 tx=0x7f1f7c007a90 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 msgr2=0x7f1f880a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f1f7801c840 tx=0x7f1f7801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 shutdown_connections 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1f740355c0 0x7f1f74037a80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 --2- 172.21.15.96:0/1909475300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f8809d690 0x7f1f880a6b60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 >> 172.21.15.96:0/1909475300 conn(0x7f1f8806bc60 msgr2=0x7f1f881bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 shutdown_connections 2024-09-13T17:44:12.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.620+0000 7f1f8e325640 1 -- 172.21.15.96:0/1909475300 wait complete. 2024-09-13T17:44:12.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:12.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.752+0000 7f419d4a9640 1 Processor -- start 2024-09-13T17:44:12.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.752+0000 7f419d4a9640 1 -- start start 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f419d4a9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4198134e70 0x7f419809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f419d4a9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419809ed60 con 0x7f4198135250 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f419d4a9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419809ddc0 con 0x7f4198134e70 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4196ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4198134e70 0x7f419809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f41967fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4198135250 0x7f4198135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38772/0 (socket says 172.21.15.96:38772) 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f41967fc640 1 -- 172.21.15.96:0/1136211464 learned_addr learned my addr 172.21.15.96:0/1136211464 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4196ffd640 1 -- 172.21.15.96:0/1136211464 >> v1:172.21.15.96:6789/0 conn(0x7f4198135250 legacy=0x7f4198135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4196ffd640 1 -- 172.21.15.96:0/1136211464 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41980a68b0 con 0x7f4198134e70 2024-09-13T17:44:12.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4196ffd640 1 --2- 172.21.15.96:0/1136211464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419809e820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4190009d90 tx=0x7f419001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c78f903226ddc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4195ffb640 1 -- 172.21.15.96:0/1136211464 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41900a57b0 con 0x7f4198134e70 2024-09-13T17:44:12.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4195ffb640 1 -- 172.21.15.96:0/1136211464 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41900a5910 con 0x7f4198134e70 2024-09-13T17:44:12.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.756+0000 7f4195ffb640 1 -- 172.21.15.96:0/1136211464 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41900a5bd0 con 0x7f4198134e70 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1136211464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 msgr2=0x7f419809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1136211464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419809e820 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f4190009d90 tx=0x7f419001c970 comp rx=0 tx=0).stop 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1136211464 shutdown_connections 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1136211464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419809e820 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1136211464 >> 172.21.15.96:0/1136211464 conn(0x7f4198098a60 msgr2=0x7f419809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1136211464 shutdown_connections 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1136211464 wait complete. 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 Processor -- start 2024-09-13T17:44:12.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- start start 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419813b3e0 con 0x7f4198134e70 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f4196ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55150/0 (socket says 172.21.15.96:55150) 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f4196ffd640 1 -- 172.21.15.96:0/1629802415 learned_addr learned my addr 172.21.15.96:0/1629802415 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f4196ffd640 1 -- 172.21.15.96:0/1629802415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4190009a70 con 0x7f4198134e70 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f4196ffd640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f41900b0040 tx=0x7f4190019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41900b9790 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41900b98f0 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41980a6230 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41900b9bd0 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41980a07c0 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41981368b0 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f41900b9d30 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 0x7f4178037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f41900e3120 con 0x7f4198134e70 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.760+0000 7f41967fc640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 0x7f4178037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:12.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.764+0000 7f41967fc640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 0x7f4178037a80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4180001610 tx=0x7f4180007ad0 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:12.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.768+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41900abbc0 con 0x7f4198134e70 2024-09-13T17:44:12.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41980a07c0 con 0x7f4198134e70 2024-09-13T17:44:12.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f418f7fe640 1 -- 172.21.15.96:0/1629802415 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41900b2020 con 0x7f4198134e70 2024-09-13T17:44:12.916 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:12.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 msgr2=0x7f4178037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 0x7f4178037a80 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4180001610 tx=0x7f4180007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 msgr2=0x7f419813aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f41900b0040 tx=0x7f4190019710 comp rx=0 tx=0).stop 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 shutdown_connections 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f41780355c0 0x7f4178037a80 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 --2- 172.21.15.96:0/1629802415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4198134e70 0x7f419813aea0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 >> 172.21.15.96:0/1629802415 conn(0x7f4198098a60 msgr2=0x7f41980993f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 shutdown_connections 2024-09-13T17:44:12.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:12.912+0000 7f419d4a9640 1 -- 172.21.15.96:0/1629802415 wait complete. 2024-09-13T17:44:12.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:13.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f29723ef640 1 Processor -- start 2024-09-13T17:44:13.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f29723ef640 1 -- start start 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f29723ef640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f296c134e70 0x7f296c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f29723ef640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c09ed60 con 0x7f296c135250 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f29723ef640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c09ddc0 con 0x7f296c134e70 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f296bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f296c134e70 0x7f296c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f296b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f296c135250 0x7f296c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38788/0 (socket says 172.21.15.96:38788) 2024-09-13T17:44:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.040+0000 7f296b7fe640 1 -- 172.21.15.96:0/4018207856 learned_addr learned my addr 172.21.15.96:0/4018207856 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296bfff640 1 -- 172.21.15.96:0/4018207856 >> v1:172.21.15.96:6789/0 conn(0x7f296c135250 legacy=0x7f296c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296bfff640 1 -- 172.21.15.96:0/4018207856 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0a68b0 con 0x7f296c134e70 2024-09-13T17:44:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296affd640 1 -- 172.21.15.96:0/4018207856 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635549740 0 0) 0x7f2960001120 con 0x7f296c135250 2024-09-13T17:44:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296bfff640 1 --2- 172.21.15.96:0/4018207856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c09e820 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f295c009d90 tx=0x7f295c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a789394ba9e01c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296affd640 1 -- 172.21.15.96:0/4018207856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f295c0a57b0 con 0x7f296c134e70 2024-09-13T17:44:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296affd640 1 -- 172.21.15.96:0/4018207856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0a5910 con 0x7f296c134e70 2024-09-13T17:44:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f296affd640 1 -- 172.21.15.96:0/4018207856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f295c0a5bd0 con 0x7f296c134e70 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 -- 172.21.15.96:0/4018207856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 msgr2=0x7f296c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 --2- 172.21.15.96:0/4018207856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c09e820 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f295c009d90 tx=0x7f295c01c970 comp rx=0 tx=0).stop 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 -- 172.21.15.96:0/4018207856 shutdown_connections 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 --2- 172.21.15.96:0/4018207856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c09e820 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 -- 172.21.15.96:0/4018207856 >> 172.21.15.96:0/4018207856 conn(0x7f296c098a60 msgr2=0x7f296c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 -- 172.21.15.96:0/4018207856 shutdown_connections 2024-09-13T17:44:13.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.044+0000 7f29723ef640 1 -- 172.21.15.96:0/4018207856 wait complete. 2024-09-13T17:44:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 Processor -- start 2024-09-13T17:44:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 -- start start 2024-09-13T17:44:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c13bb90 con 0x7f296c134e70 2024-09-13T17:44:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55172/0 (socket says 172.21.15.96:55172) 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296bfff640 1 -- 172.21.15.96:0/886218651 learned_addr learned my addr 172.21.15.96:0/886218651 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296bfff640 1 -- 172.21.15.96:0/886218651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c009a70 con 0x7f296c134e70 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296bfff640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f295c0b0040 tx=0x7f295c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f295c0b8690 con 0x7f296c134e70 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0b87f0 con 0x7f296c134e70 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c0a2e20 con 0x7f296c134e70 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f295c0b8a90 con 0x7f296c134e70 2024-09-13T17:44:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0a1b90 con 0x7f296c134e70 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f295c0b4040 con 0x7f296c134e70 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c0a2c50 con 0x7f296c134e70 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 0x7f2958037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f295c0e2f30 con 0x7f296c134e70 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296b7fe640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 0x7f2958037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.048+0000 7f296b7fe640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 0x7f2958037a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2960000c10 tx=0x7f2960001320 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.052+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f295c0caba0 con 0x7f296c134e70 2024-09-13T17:44:13.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.204+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f296c1368b0 con 0x7f296c134e70 2024-09-13T17:44:13.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.204+0000 7f2968ff9640 1 -- 172.21.15.96:0/886218651 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f295c0b2070 con 0x7f296c134e70 2024-09-13T17:44:13.209 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 msgr2=0x7f2958037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 0x7f2958037a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2960000c10 tx=0x7f2960001320 comp rx=0 tx=0).stop 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 msgr2=0x7f296c0a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f295c0b0040 tx=0x7f295c019710 comp rx=0 tx=0).stop 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 shutdown_connections 2024-09-13T17:44:13.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f29580355c0 0x7f2958037a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 --2- 172.21.15.96:0/886218651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f296c134e70 0x7f296c0a25f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 >> 172.21.15.96:0/886218651 conn(0x7f296c098a60 msgr2=0x7f296c131da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 shutdown_connections 2024-09-13T17:44:13.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.208+0000 7f29723ef640 1 -- 172.21.15.96:0/886218651 wait complete. 2024-09-13T17:44:13.227 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:13.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1de853640 1 Processor -- start 2024-09-13T17:44:13.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1de853640 1 -- start start 2024-09-13T17:44:13.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1de853640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1d8134670 0x7fd1d8134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1de853640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1d8071670 con 0x7fd1d806e5c0 2024-09-13T17:44:13.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1de853640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1d8071360 con 0x7fd1d8134670 2024-09-13T17:44:13.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d77fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd1d806e5c0 0x7fd1d806ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38796/0 (socket says 172.21.15.96:38796) 2024-09-13T17:44:13.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d77fe640 1 -- 172.21.15.96:0/3706313952 learned_addr learned my addr 172.21.15.96:0/3706313952 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:13.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d7fff640 1 --2- 172.21.15.96:0/3706313952 >> v2:172.21.15.96:3300/0 conn(0x7fd1d8134670 0x7fd1d8134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d7fff640 1 -- 172.21.15.96:0/3706313952 >> v1:172.21.15.96:6789/0 conn(0x7fd1d806e5c0 legacy=0x7fd1d806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d6ffd640 1 -- 172.21.15.96:0/3706313952 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343157120 0 0) 0x7fd1d8071670 con 0x7fd1d806e5c0 2024-09-13T17:44:13.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.340+0000 7fd1d7fff640 1 -- 172.21.15.96:0/3706313952 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80a7200 con 0x7fd1d8134670 2024-09-13T17:44:13.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 --2- 172.21.15.96:0/3706313952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d8134670 0x7fd1d8134a50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009740 tx=0x7fd1c801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b6a682c4012fb44 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d6ffd640 1 -- 172.21.15.96:0/3706313952 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c80ab530 con 0x7fd1d8134670 2024-09-13T17:44:13.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d6ffd640 1 -- 172.21.15.96:0/3706313952 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c80a7050 con 0x7fd1d8134670 2024-09-13T17:44:13.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d6ffd640 1 -- 172.21.15.96:0/3706313952 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c80ab7e0 con 0x7fd1d8134670 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- 172.21.15.96:0/3706313952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d8134670 msgr2=0x7fd1d8134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 --2- 172.21.15.96:0/3706313952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d8134670 0x7fd1d8134a50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009740 tx=0x7fd1c801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- 172.21.15.96:0/3706313952 shutdown_connections 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 --2- 172.21.15.96:0/3706313952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d8134670 0x7fd1d8134a50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- 172.21.15.96:0/3706313952 >> 172.21.15.96:0/3706313952 conn(0x7fd1d8098a60 msgr2=0x7fd1d809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- 172.21.15.96:0/3706313952 shutdown_connections 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- 172.21.15.96:0/3706313952 wait complete. 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 Processor -- start 2024-09-13T17:44:13.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- start start 2024-09-13T17:44:13.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1de853640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1d80a0000 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55200/0 (socket says 172.21.15.96:55200) 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 -- 172.21.15.96:0/3318395610 learned_addr learned my addr 172.21.15.96:0/3318395610 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 -- 172.21.15.96:0/3318395610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c8009420 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d7fff640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8018fa0 tx=0x7fd1c801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c80b7660 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c80b77e0 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.344+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1c80b7ac0 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d8138720 con 0x7fd1d806e5c0 2024-09-13T17:44:13.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1d8138e00 con 0x7fd1d806e5c0 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fd1c80b7c20 con 0x7fd1d806e5c0 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1d809d520 con 0x7fd1d806e5c0 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1d4ff9640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 0x7fd1c4037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fd1c80e2a70 con 0x7fd1d806e5c0 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1d77fe640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 0x7fd1c4037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.348+0000 7fd1d77fe640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 0x7fd1c4037a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc007fd0 tx=0x7fd1cc007920 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.352+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1c80e4e60 con 0x7fd1d806e5c0 2024-09-13T17:44:13.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1d806a1b0 con 0x7fd1d806e5c0 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1d4ff9640 1 -- 172.21.15.96:0/3318395610 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v42) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd1c80bb020 con 0x7fd1d806e5c0 2024-09-13T17:44:13.504 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:13.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 msgr2=0x7fd1c4037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 0x7fd1c4037a80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc007fd0 tx=0x7fd1cc007920 comp rx=0 tx=0).stop 2024-09-13T17:44:13.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 msgr2=0x7fd1d81372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.506 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8018fa0 tx=0x7fd1c801abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 shutdown_connections 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd1c40355c0 0x7fd1c4037a80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 --2- 172.21.15.96:0/3318395610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1d806e5c0 0x7fd1d81372c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 >> 172.21.15.96:0/3318395610 conn(0x7fd1d8098a60 msgr2=0x7fd1d8131f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 shutdown_connections 2024-09-13T17:44:13.507 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.500+0000 7fd1de853640 1 -- 172.21.15.96:0/3318395610 wait complete. 2024-09-13T17:44:13.521 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:13.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e3924640 1 Processor -- start 2024-09-13T17:44:13.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e3924640 1 -- start start 2024-09-13T17:44:13.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e3924640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd6dc0a1370 0x7fd6dc0a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e3924640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6dc09eca0 con 0x7fd6dc0a1c90 2024-09-13T17:44:13.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e3924640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6dc09d440 con 0x7fd6dc0a1370 2024-09-13T17:44:13.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e16c0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd6dc0a1370 0x7fd6dc0a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e0ebf640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd6dc0a1c90 0x7fd6dc1381e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38802/0 (socket says 172.21.15.96:38802) 2024-09-13T17:44:13.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e16c0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd6dc0a1370 0x7fd6dc0a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55210/0 (socket says 172.21.15.96:55210) 2024-09-13T17:44:13.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.628+0000 7fd6e0ebf640 1 -- 172.21.15.96:0/770190913 learned_addr learned my addr 172.21.15.96:0/770190913 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:13.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e16c0640 1 -- 172.21.15.96:0/770190913 >> v1:172.21.15.96:6789/0 conn(0x7fd6dc0a1c90 legacy=0x7fd6dc1381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6dbfff640 1 -- 172.21.15.96:0/770190913 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69556593 0 0) 0x7fd6d0001120 con 0x7fd6dc0a1c90 2024-09-13T17:44:13.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e16c0640 1 -- 172.21.15.96:0/770190913 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6dc0a28b0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e16c0640 1 --2- 172.21.15.96:0/770190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc0a1750 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4b32f448d35b0b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6dbfff640 1 -- 172.21.15.96:0/770190913 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40a65d0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6dbfff640 1 -- 172.21.15.96:0/770190913 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40a6730 con 0x7fd6dc0a1370 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6dbfff640 1 -- 172.21.15.96:0/770190913 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40a69f0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 -- 172.21.15.96:0/770190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 msgr2=0x7fd6dc0a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 --2- 172.21.15.96:0/770190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc0a1750 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d401c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 -- 172.21.15.96:0/770190913 shutdown_connections 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 --2- 172.21.15.96:0/770190913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc0a1750 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 -- 172.21.15.96:0/770190913 >> 172.21.15.96:0/770190913 conn(0x7fd6dc0a6b40 msgr2=0x7fd6dc0a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 -- 172.21.15.96:0/770190913 shutdown_connections 2024-09-13T17:44:13.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.632+0000 7fd6e3924640 1 -- 172.21.15.96:0/770190913 wait complete. 2024-09-13T17:44:13.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 Processor -- start 2024-09-13T17:44:13.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 -- start start 2024-09-13T17:44:13.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6dc11ea40 con 0x7fd6dc0a1370 2024-09-13T17:44:13.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e16c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e16c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55212/0 (socket says 172.21.15.96:55212) 2024-09-13T17:44:13.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e16c0640 1 -- 172.21.15.96:0/3034391365 learned_addr learned my addr 172.21.15.96:0/3034391365 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e16c0640 1 -- 172.21.15.96:0/3034391365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4009a70 con 0x7fd6dc0a1370 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e16c0640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40a75b0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40a7710 con 0x7fd6dc0a1370 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc071a70 con 0x7fd6dc0a1370 2024-09-13T17:44:13.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6d40a79d0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6dc11f4e0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6dc065ee0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fd6d40a7b30 con 0x7fd6dc0a1370 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 0x7fd6c8037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fd6d40b7070 con 0x7fd6dc0a1370 2024-09-13T17:44:13.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e0ebf640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 0x7fd6c8037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.636+0000 7fd6e0ebf640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 0x7fd6c8037a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005890 tx=0x7fd6d0001280 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.648 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.640+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd6d40a7de0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.796+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7fd6dc11d480 con 0x7fd6dc0a1370 2024-09-13T17:44:13.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6d9ffb640 1 -- 172.21.15.96:0/3034391365 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v42) v1 ==== 102+0+11 (secure 0 0 0) 0x7fd6d40a5bc0 con 0x7fd6dc0a1370 2024-09-13T17:44:13.803 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:13.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 msgr2=0x7fd6c8037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 0x7fd6c8037a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0005890 tx=0x7fd6d0001280 comp rx=0 tx=0).stop 2024-09-13T17:44:13.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 msgr2=0x7fd6dc120eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009d90 tx=0x7fd6d4019680 comp rx=0 tx=0).stop 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 shutdown_connections 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd6c80355c0 0x7fd6c8037a80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 --2- 172.21.15.96:0/3034391365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6dc0a1370 0x7fd6dc120eb0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 >> 172.21.15.96:0/3034391365 conn(0x7fd6dc0a6b40 msgr2=0x7fd6dc0a3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 shutdown_connections 2024-09-13T17:44:13.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.800+0000 7fd6e3924640 1 -- 172.21.15.96:0/3034391365 wait complete. 2024-09-13T17:44:13.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:44:13.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda33dff640 1 Processor -- start 2024-09-13T17:44:13.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda33dff640 1 -- start start 2024-09-13T17:44:13.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda33dff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fda2c09e820 0x7fda2c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda33dff640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0a68b0 con 0x7fda2c134e70 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda33dff640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c09ddc0 con 0x7fda2c09e820 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda3139a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fda2c09e820 0x7fda2c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda31b9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fda2c134e70 0x7fda2c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38808/0 (socket says 172.21.15.96:38808) 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda31b9b640 1 -- 172.21.15.96:0/3426037466 learned_addr learned my addr 172.21.15.96:0/3426037466 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda3139a640 1 -- 172.21.15.96:0/3426037466 >> v1:172.21.15.96:6789/0 conn(0x7fda2c134e70 legacy=0x7fda2c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda3139a640 1 -- 172.21.15.96:0/3426037466 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c136970 con 0x7fda2c09e820 2024-09-13T17:44:13.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.932+0000 7fda30b99640 1 -- 172.21.15.96:0/3426037466 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 445684469 0 0) 0x7fda2c0a68b0 con 0x7fda2c134e70 2024-09-13T17:44:13.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda3139a640 1 --2- 172.21.15.96:0/3426037466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c13ae50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fda20009740 tx=0x7fda2001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94ba3ae84ddc4d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda30b99640 1 -- 172.21.15.96:0/3426037466 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda200ab530 con 0x7fda2c09e820 2024-09-13T17:44:13.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda30b99640 1 -- 172.21.15.96:0/3426037466 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda200a7050 con 0x7fda2c09e820 2024-09-13T17:44:13.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda30b99640 1 -- 172.21.15.96:0/3426037466 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda200ab7e0 con 0x7fda2c09e820 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/3426037466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 msgr2=0x7fda2c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 --2- 172.21.15.96:0/3426037466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c13ae50 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fda20009740 tx=0x7fda2001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/3426037466 shutdown_connections 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 --2- 172.21.15.96:0/3426037466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c13ae50 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/3426037466 >> 172.21.15.96:0/3426037466 conn(0x7fda2c098a60 msgr2=0x7fda2c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/3426037466 shutdown_connections 2024-09-13T17:44:13.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/3426037466 wait complete. 2024-09-13T17:44:13.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 Processor -- start 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- start start 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c13a530 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda31b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda31b9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55220/0 (socket says 172.21.15.96:55220) 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda31b9b640 1 -- 172.21.15.96:0/701720159 learned_addr learned my addr 172.21.15.96:0/701720159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda31b9b640 1 -- 172.21.15.96:0/701720159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda20009420 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda31b9b640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c840 tx=0x7fda2400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda2400a660 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2400a7e0 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c0a3ec0 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fda2400aac0 con 0x7fda2c09e820 2024-09-13T17:44:13.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.936+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c071600 con 0x7fda2c09e820 2024-09-13T17:44:13.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7fda2400ac20 con 0x7fda2c09e820 2024-09-13T17:44:13.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c071370 con 0x7fda2c09e820 2024-09-13T17:44:13.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda2a7fc640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 0x7fda18037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:13.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fda24047650 con 0x7fda2c09e820 2024-09-13T17:44:13.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda3139a640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 0x7fda18037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:13.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.940+0000 7fda3139a640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 0x7fda18037a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fda200ad000 tx=0x7fda20008040 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:13.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:13.944+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fda2401a6b0 con 0x7fda2c09e820 2024-09-13T17:44:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.100+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda2c0a4360 con 0x7fda2c09e820 2024-09-13T17:44:14.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.100+0000 7fda2a7fc640 1 -- 172.21.15.96:0/701720159 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v42) v1 ==== 80+0+467 (secure 0 0 0) 0x7fda2401a2c0 con 0x7fda2c09e820 2024-09-13T17:44:14.106 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2785343960 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3263021005 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2962687900 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/715245426 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3193678716 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3614824494 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:14.107 INFO:teuthology.orchestra.run.smithi096.stderr:listed 8 entries 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 msgr2=0x7fda18037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 0x7fda18037a80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fda200ad000 tx=0x7fda20008040 comp rx=0 tx=0).stop 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 msgr2=0x7fda2c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fda2400c840 tx=0x7fda2400cd00 comp rx=0 tx=0).stop 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 shutdown_connections 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fda180355c0 0x7fda18037a80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 --2- 172.21.15.96:0/701720159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda2c09e820 0x7fda2c06f6d0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 >> 172.21.15.96:0/701720159 conn(0x7fda2c098a60 msgr2=0x7fda2c134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:14.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 shutdown_connections 2024-09-13T17:44:14.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.104+0000 7fda33dff640 1 -- 172.21.15.96:0/701720159 wait complete. 2024-09-13T17:44:14.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:44:14.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:14.134 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[39400]: fuse finished with error 0 and tester_r 0 2024-09-13T17:44:14.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:14.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:44:14.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.169 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:44:14.219 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:44:14.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.376+0000 7f4052532640 1 Processor -- start 2024-09-13T17:44:14.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.376+0000 7f4052532640 1 -- start start 2024-09-13T17:44:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f404c134e70 0x7f404c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f404c09ed60 con 0x7f404c135250 2024-09-13T17:44:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f404c09ddc0 con 0x7f404c134e70 2024-09-13T17:44:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f404c134e70 0x7f404c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f404c135250 0x7f404c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38824/0 (socket says 172.21.15.96:38824) 2024-09-13T17:44:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404b7fe640 1 -- 172.21.15.96:0/4042974074 learned_addr learned my addr 172.21.15.96:0/4042974074 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404bfff640 1 -- 172.21.15.96:0/4042974074 >> v1:172.21.15.96:6789/0 conn(0x7f404c135250 legacy=0x7f404c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404bfff640 1 -- 172.21.15.96:0/4042974074 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c0a68b0 con 0x7f404c134e70 2024-09-13T17:44:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404bfff640 1 --2- 172.21.15.96:0/4042974074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c09e820 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4040009740 tx=0x7f404001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba363749923109b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404affd640 1 -- 172.21.15.96:0/4042974074 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40400ab530 con 0x7f404c134e70 2024-09-13T17:44:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404affd640 1 -- 172.21.15.96:0/4042974074 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40400a7050 con 0x7f404c134e70 2024-09-13T17:44:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f404affd640 1 -- 172.21.15.96:0/4042974074 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40400ab7e0 con 0x7f404c134e70 2024-09-13T17:44:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- 172.21.15.96:0/4042974074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 msgr2=0x7f404c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 --2- 172.21.15.96:0/4042974074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c09e820 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4040009740 tx=0x7f404001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- 172.21.15.96:0/4042974074 shutdown_connections 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 --2- 172.21.15.96:0/4042974074 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c09e820 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- 172.21.15.96:0/4042974074 >> 172.21.15.96:0/4042974074 conn(0x7f404c098a60 msgr2=0x7f404c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- 172.21.15.96:0/4042974074 shutdown_connections 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.380+0000 7f4052532640 1 -- 172.21.15.96:0/4042974074 wait complete. 2024-09-13T17:44:14.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 Processor -- start 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 -- start start 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c11c930 con 0x7f404c134e70 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55242/0 (socket says 172.21.15.96:55242) 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404bfff640 1 -- 172.21.15.96:0/3744507629 learned_addr learned my addr 172.21.15.96:0/3744507629 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404bfff640 1 -- 172.21.15.96:0/3744507629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040009420 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404bfff640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f404001a050 tx=0x7f40400191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40400abea0 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40400b4410 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f404c11cb60 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40400b46b0 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f404c0a5120 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404c13b4b0 con 0x7f404c134e70 2024-09-13T17:44:14.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f40400b2040 con 0x7f404c134e70 2024-09-13T17:44:14.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 0x7f4038037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:14.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f40400e1fb0 con 0x7f404c134e70 2024-09-13T17:44:14.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404b7fe640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 0x7f4038037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:14.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.384+0000 7f404b7fe640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 0x7f4038037a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f403c001610 tx=0x7f403c007ad0 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:14.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.388+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4040113db0 con 0x7f404c134e70 2024-09-13T17:44:14.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.568+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f404c13af00 con 0x7f404c134e70 2024-09-13T17:44:14.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.568+0000 7f4048ff9640 1 -- 172.21.15.96:0/3744507629 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+152 (secure 0 0 0) 0x7f40400b0030 con 0x7f404c134e70 2024-09-13T17:44:14.573 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:44:14.573 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs_b, metadata pool: cephfs_b_metadata, data pools: [cephfs_b_data ] 2024-09-13T17:44:14.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 msgr2=0x7f4038037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 0x7f4038037a80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f403c001610 tx=0x7f403c007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 msgr2=0x7f404c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f404001a050 tx=0x7f40400191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 shutdown_connections 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f40380355c0 0x7f4038037a80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 --2- 172.21.15.96:0/3744507629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f404c134e70 0x7f404c06f6d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 >> 172.21.15.96:0/3744507629 conn(0x7f404c098a60 msgr2=0x7f404c133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 shutdown_connections 2024-09-13T17:44:14.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:14.572+0000 7f4052532640 1 -- 172.21.15.96:0/3744507629 wait complete. 2024-09-13T17:44:14.591 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-13T17:44:14.591 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:44:14.591 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:44:14.591 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:44:14.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:14.592 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-13T17:44:14.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:44:14.592 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:44:14.592 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:14.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.593 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:44:14.601 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.602 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:44:14.603 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.603 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:44:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.603 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42428sec preferred_lft 42428sec 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.604 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.605 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.606 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:44:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:44:14.607 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:14.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.607 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:44:14.653 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:44:14.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.655 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:44:14.707 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:44:14.707 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:44:14.707 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:44:14.707 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:44:14.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.708 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:44:14.709 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:44:14.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:44:14.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:44:14.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:44:14.709 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:44:14.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.783 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:44:14.831 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:44:14.831 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:14.831 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:14.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:14.831 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:44:14.891 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:44:14.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:14.892 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:14.939 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:14.939 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:14.940 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:14.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:15.045 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:15.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:15.100 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:15.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:15.101 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:15.147 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:15.148 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:44:15.148 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_fs=cephfs) 2024-09-13T17:44:15.192 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:15.207 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:15.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:15.265 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:15.266 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:15.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:15.267 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:15.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:44:15.268+0000 7feed07cd580 -1 init, newargv = 0x55e886643810 newargc=15 2024-09-13T17:44:15.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[40000]: starting ceph client 2024-09-13T17:44:15.275 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:15.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[40000]: starting fuse 2024-09-13T17:44:16.378 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:16.394 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:16.394 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:16.451 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:16.452 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:16.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:16.452 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:16.500 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:16.500 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:44:16.500 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:16.501 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:44:16.502 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:16.622 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.40000.asok 2024-09-13T17:44:16.804 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40000.asok 2024-09-13T17:44:16.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:16.804 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40000.asok status 2024-09-13T17:44:16.963 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "40000", 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:44:17.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 4980, 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 4980 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1600694241 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:17.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 1600694241 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.4980 192.168.0.2:0/1600694241", 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/1600694241", 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 20, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 42, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:44:17.027 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:17.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:17.029 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:44:17.029 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:17.035 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:44:17.035 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:17.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:17.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:17.091 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49e5ba2640 1 Processor -- start 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49e5ba2640 1 -- start start 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49e5ba2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49e006f6d0 0x7f49e0134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49e5ba2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e006e130 con 0x7f49e009dc20 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49e5ba2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e00a6750 con 0x7f49e006f6d0 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49df7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f49e006f6d0 0x7f49e0134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49deffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f49e009dc20 0x7f49e0134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38840/0 (socket says 172.21.15.96:38840) 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49deffd640 1 -- 172.21.15.96:0/761147779 learned_addr learned my addr 172.21.15.96:0/761147779 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:17.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49df7fe640 1 --2- 172.21.15.96:0/761147779 >> v2:172.21.15.96:3300/0 conn(0x7f49e006f6d0 0x7f49e0134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55248/0 (socket says 172.21.15.96:55248) 2024-09-13T17:44:17.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49df7fe640 1 -- 172.21.15.96:0/761147779 >> v1:172.21.15.96:6789/0 conn(0x7f49e009dc20 legacy=0x7f49e0134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49df7fe640 1 -- 172.21.15.96:0/761147779 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00712e0 con 0x7f49e006f6d0 2024-09-13T17:44:17.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.256+0000 7f49de7fc640 1 -- 172.21.15.96:0/761147779 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146755895 0 0) 0x7f49e006e130 con 0x7f49e009dc20 2024-09-13T17:44:17.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49df7fe640 1 --2- 172.21.15.96:0/761147779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0134ab0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009740 tx=0x7f49d001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6ba8ceec8490f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49de7fc640 1 -- 172.21.15.96:0/761147779 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00ab400 con 0x7f49e006f6d0 2024-09-13T17:44:17.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49de7fc640 1 -- 172.21.15.96:0/761147779 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d00a7050 con 0x7f49e006f6d0 2024-09-13T17:44:17.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49de7fc640 1 -- 172.21.15.96:0/761147779 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00ab6b0 con 0x7f49e006f6d0 2024-09-13T17:44:17.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- 172.21.15.96:0/761147779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 msgr2=0x7f49e0134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:17.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/761147779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0134ab0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009740 tx=0x7f49d001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:17.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- 172.21.15.96:0/761147779 shutdown_connections 2024-09-13T17:44:17.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/761147779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0134ab0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:17.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- 172.21.15.96:0/761147779 >> 172.21.15.96:0/761147779 conn(0x7f49e0098a60 msgr2=0x7f49e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:17.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- 172.21.15.96:0/761147779 shutdown_connections 2024-09-13T17:44:17.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- 172.21.15.96:0/761147779 wait complete. 2024-09-13T17:44:17.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 Processor -- start 2024-09-13T17:44:17.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- start start 2024-09-13T17:44:17.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49e5ba2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e0134080 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55254/0 (socket says 172.21.15.96:55254) 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.260+0000 7f49df7fe640 1 -- 172.21.15.96:0/2744738163 learned_addr learned my addr 172.21.15.96:0/2744738163 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49df7fe640 1 -- 172.21.15.96:0/2744738163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009420 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49df7fe640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f49d0019fc0 tx=0x7f49d001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00b7450 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d00b75b0 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49d00b7850 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e00a40b0 con 0x7f49e006f6d0 2024-09-13T17:44:17.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e00a2fc0 con 0x7f49e006f6d0 2024-09-13T17:44:17.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 20) v1 ==== 43418+0+0 (secure 0 0 0) 0x7f49d00b79b0 con 0x7f49e006f6d0 2024-09-13T17:44:17.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e00a32b0 con 0x7f49e006f6d0 2024-09-13T17:44:17.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 0x7f49c0037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:17.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f49d00e2900 con 0x7f49e006f6d0 2024-09-13T17:44:17.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49deffd640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 0x7f49c0037a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:17.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.264+0000 7f49deffd640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 0x7f49c0037a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49c8007fd0 tx=0x7f49c8007920 comp rx=0 tx=0).ready entity=mgr.4821 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:17.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.268+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49d00c78f0 con 0x7f49e006f6d0 2024-09-13T17:44:17.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:17.464+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) v1 -- 0x7f49e0130a70 con 0x7f49e006f6d0 2024-09-13T17:44:18.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.084+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f49d00b5540 con 0x7f49e006f6d0 2024-09-13T17:44:18.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.084+0000 7f49d7fff640 1 -- 172.21.15.96:0/2744738163 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v21) v1 ==== 85+0+0 (secure 0 0 0) 0x7f49d00aa7a0 con 0x7f49e006f6d0 2024-09-13T17:44:18.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 msgr2=0x7f49c0037a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 0x7f49c0037a80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f49c8007fd0 tx=0x7f49c8007920 comp rx=0 tx=0).stop 2024-09-13T17:44:18.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 msgr2=0x7f49e0098e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f49d0019fc0 tx=0x7f49d001abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 shutdown_connections 2024-09-13T17:44:18.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f49c00355c0 0x7f49c0037a80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 --2- 172.21.15.96:0/2744738163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49e006f6d0 0x7f49e0098e40 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 >> 172.21.15.96:0/2744738163 conn(0x7f49e0098a60 msgr2=0x7f49e006fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 shutdown_connections 2024-09-13T17:44:18.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.088+0000 7f49e5ba2640 1 -- 172.21.15.96:0/2744738163 wait complete. 2024-09-13T17:44:18.110 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:44:18.110 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:44:18.176 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:44:18.176 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:18.176 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:18.181 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:18.212 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:18.208+0000 7feb80efc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:44:18.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.324+0000 7fd664fac640 1 Processor -- start 2024-09-13T17:44:18.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.324+0000 7fd664fac640 1 -- start start 2024-09-13T17:44:18.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd660136100 0x7fd66009d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd66009d090 con 0x7fd6601364e0 2024-09-13T17:44:18.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66009cd80 con 0x7fd660136100 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65e59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd660136100 0x7fd66009d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65dd9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd6601364e0 0x7fd660099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38862/0 (socket says 172.21.15.96:38862) 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65dd9b640 1 -- 172.21.15.96:0/3889414466 learned_addr learned my addr 172.21.15.96:0/3889414466 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65e59c640 1 -- 172.21.15.96:0/3889414466 >> v1:172.21.15.96:6789/0 conn(0x7fd6601364e0 legacy=0x7fd660099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65d59a640 1 -- 172.21.15.96:0/3889414466 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196253264 0 0) 0x7fd66009d090 con 0x7fd6601364e0 2024-09-13T17:44:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65e59c640 1 -- 172.21.15.96:0/3889414466 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66009dec0 con 0x7fd660136100 2024-09-13T17:44:18.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65e59c640 1 --2- 172.21.15.96:0/3889414466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd66009d7e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd648009740 tx=0x7fd64801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae90bff08835f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65d59a640 1 -- 172.21.15.96:0/3889414466 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6480ac540 con 0x7fd660136100 2024-09-13T17:44:18.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65d59a640 1 -- 172.21.15.96:0/3889414466 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480ac6a0 con 0x7fd660136100 2024-09-13T17:44:18.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd65d59a640 1 -- 172.21.15.96:0/3889414466 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6480ac960 con 0x7fd660136100 2024-09-13T17:44:18.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- 172.21.15.96:0/3889414466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 msgr2=0x7fd66009d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 --2- 172.21.15.96:0/3889414466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd66009d7e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd648009740 tx=0x7fd64801c350 comp rx=0 tx=0).stop 2024-09-13T17:44:18.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- 172.21.15.96:0/3889414466 shutdown_connections 2024-09-13T17:44:18.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 --2- 172.21.15.96:0/3889414466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd66009d7e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- 172.21.15.96:0/3889414466 >> 172.21.15.96:0/3889414466 conn(0x7fd66006f270 msgr2=0x7fd660071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- 172.21.15.96:0/3889414466 shutdown_connections 2024-09-13T17:44:18.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.328+0000 7fd664fac640 1 -- 172.21.15.96:0/3889414466 wait complete. 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 Processor -- start 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 -- start start 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600a11c0 con 0x7fd660136100 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55270/0 (socket says 172.21.15.96:55270) 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65e59c640 1 -- 172.21.15.96:0/854426361 learned_addr learned my addr 172.21.15.96:0/854426361 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65e59c640 1 -- 172.21.15.96:0/854426361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd648009420 con 0x7fd660136100 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65e59c640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd648019f80 tx=0x7fd64801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6480b8580 con 0x7fd660136100 2024-09-13T17:44:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6480b86e0 con 0x7fd660136100 2024-09-13T17:44:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6600a1350 con 0x7fd660136100 2024-09-13T17:44:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd6480b89a0 con 0x7fd660136100 2024-09-13T17:44:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6600a2c10 con 0x7fd660136100 2024-09-13T17:44:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd66009d7e0 con 0x7fd660136100 2024-09-13T17:44:18.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fd6480b8b00 con 0x7fd660136100 2024-09-13T17:44:18.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 0x7fd640037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fd6480e28f0 con 0x7fd660136100 2024-09-13T17:44:18.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65dd9b640 1 -- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 msgr2=0x7fd640037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:18.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.332+0000 7fd65dd9b640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 0x7fd640037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:18.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.336+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd6480b8db0 con 0x7fd660136100 2024-09-13T17:44:18.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.484+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd66006a2d0 con 0x7fd660136100 2024-09-13T17:44:18.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd656ffd640 1 -- 172.21.15.96:0/854426361 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14765 (secure 0 0 0) 0x7fd6480b1070 con 0x7fd660136100 2024-09-13T17:44:18.492 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:18.492 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:57.009683+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:18.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 msgr2=0x7fd640037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:18.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 0x7fd640037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 msgr2=0x7fd6600a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd648019f80 tx=0x7fd64801ab90 comp rx=0 tx=0).stop 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 shutdown_connections 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd640035570 0x7fd640037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 --2- 172.21.15.96:0/854426361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd660136100 0x7fd6600a3670 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 >> 172.21.15.96:0/854426361 conn(0x7fd66006f270 msgr2=0x7fd660070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 shutdown_connections 2024-09-13T17:44:18.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.488+0000 7fd664fac640 1 -- 172.21.15.96:0/854426361 wait complete. 2024-09-13T17:44:18.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:18.511 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:18.593 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:18.588+0000 7feb80efc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:44:18.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.624+0000 7fe9c3aff640 1 Processor -- start 2024-09-13T17:44:18.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.624+0000 7fe9c3aff640 1 -- start start 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe9bc09d690 0x7fe9bc138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc138050 con 0x7fe9bc1387d0 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc09c410 con 0x7fe9bc09d690 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c189b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe9bc09d690 0x7fe9bc138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c109a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe9bc1387d0 0x7fe9bc134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38866/0 (socket says 172.21.15.96:38866) 2024-09-13T17:44:18.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c109a640 1 -- 172.21.15.96:0/3691376786 learned_addr learned my addr 172.21.15.96:0/3691376786 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c189b640 1 -- 172.21.15.96:0/3691376786 >> v1:172.21.15.96:6789/0 conn(0x7fe9bc1387d0 legacy=0x7fe9bc134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c189b640 1 -- 172.21.15.96:0/3691376786 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc09cba0 con 0x7fe9bc09d690 2024-09-13T17:44:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c189b640 1 --2- 172.21.15.96:0/3691376786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc138290 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009740 tx=0x7fe9ac01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b63cf051e01bc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c0899640 1 -- 172.21.15.96:0/3691376786 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac0a8840 con 0x7fe9bc09d690 2024-09-13T17:44:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c0899640 1 -- 172.21.15.96:0/3691376786 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0a89a0 con 0x7fe9bc09d690 2024-09-13T17:44:18.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c0899640 1 -- 172.21.15.96:0/3691376786 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac0a8c60 con 0x7fe9bc09d690 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3691376786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 msgr2=0x7fe9bc138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3691376786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc138290 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009740 tx=0x7fe9ac01ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3691376786 shutdown_connections 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3691376786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc138290 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3691376786 >> 172.21.15.96:0/3691376786 conn(0x7fe9bc06bc60 msgr2=0x7fe9bc06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3691376786 shutdown_connections 2024-09-13T17:44:18.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.628+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3691376786 wait complete. 2024-09-13T17:44:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 Processor -- start 2024-09-13T17:44:18.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 -- start start 2024-09-13T17:44:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0a5e10 con 0x7fe9bc09d690 2024-09-13T17:44:18.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c189b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55286/0 (socket says 172.21.15.96:55286) 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c189b640 1 -- 172.21.15.96:0/3664412269 learned_addr learned my addr 172.21.15.96:0/3664412269 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c189b640 1 -- 172.21.15.96:0/3664412269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac009420 con 0x7fe9bc09d690 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c189b640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac01c840 tx=0x7fe9ac01b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac0b0840 con 0x7fe9bc09d690 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac0b09a0 con 0x7fe9bc09d690 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0a70a0 con 0x7fe9bc09d690 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe9ac0b0c60 con 0x7fe9bc09d690 2024-09-13T17:44:18.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc0a4300 con 0x7fe9bc09d690 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fe9ac0b7040 con 0x7fe9bc09d690 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc09bc00 con 0x7fe9bc09d690 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 0x7fe9a00379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fe9ac0e58f0 con 0x7fe9bc09d690 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c109a640 1 -- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 msgr2=0x7fe9a00379e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:18.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.632+0000 7fe9c109a640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 0x7fe9a00379e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:18.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.636+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe9bc09bc00 con 0x7fe9bc09d690 2024-09-13T17:44:18.729 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:18.724+0000 7feb80efc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:44:18.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.796+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe9bc09bc00 con 0x7fe9bc09d690 2024-09-13T17:44:18.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.796+0000 7fe9b27fc640 1 -- 172.21.15.96:0/3664412269 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe9bc09bc00 con 0x7fe9bc09d690 2024-09-13T17:44:18.801 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:18.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 msgr2=0x7fe9a00379e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:18.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 0x7fe9a00379e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 msgr2=0x7fe9bc0a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac01c840 tx=0x7fe9ac01b9b0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 shutdown_connections 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fe9a0035520 0x7fe9a00379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 --2- 172.21.15.96:0/3664412269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe9bc09d690 0x7fe9bc0a6b60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 >> 172.21.15.96:0/3664412269 conn(0x7fe9bc06bc60 msgr2=0x7fe9bc1bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 shutdown_connections 2024-09-13T17:44:18.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.800+0000 7fe9c3aff640 1 -- 172.21.15.96:0/3664412269 wait complete. 2024-09-13T17:44:18.819 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ef505640 1 Processor -- start 2024-09-13T17:44:18.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ef505640 1 -- start start 2024-09-13T17:44:18.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ef505640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2e8137110 0x7fd2e81374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ef505640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2e809a180 con 0x7fd2e809ba80 2024-09-13T17:44:18.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ef505640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2e809adc0 con 0x7fd2e8137110 2024-09-13T17:44:18.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ed2a1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd2e8137110 0x7fd2e81374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ecaa0640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd2e809ba80 0x7fd2e8069f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38874/0 (socket says 172.21.15.96:38874) 2024-09-13T17:44:18.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ecaa0640 1 -- 172.21.15.96:0/4282188679 learned_addr learned my addr 172.21.15.96:0/4282188679 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:18.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ed2a1640 1 -- 172.21.15.96:0/4282188679 >> v1:172.21.15.96:6789/0 conn(0x7fd2e809ba80 legacy=0x7fd2e8069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ed2a1640 1 -- 172.21.15.96:0/4282188679 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8099120 con 0x7fd2e8137110 2024-09-13T17:44:18.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2dffff640 1 -- 172.21.15.96:0/4282188679 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375107274 0 0) 0x7fd2e809a180 con 0x7fd2e809ba80 2024-09-13T17:44:18.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2ed2a1640 1 --2- 172.21.15.96:0/4282188679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e8137110 0x7fd2e81374f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009740 tx=0x7fd2e401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee6cf658ada8d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2dffff640 1 -- 172.21.15.96:0/4282188679 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e40ab530 con 0x7fd2e8137110 2024-09-13T17:44:18.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2dffff640 1 -- 172.21.15.96:0/4282188679 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40a7050 con 0x7fd2e8137110 2024-09-13T17:44:18.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.928+0000 7fd2dffff640 1 -- 172.21.15.96:0/4282188679 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e40ab7e0 con 0x7fd2e8137110 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/4282188679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e8137110 msgr2=0x7fd2e81374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 --2- 172.21.15.96:0/4282188679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e8137110 0x7fd2e81374f0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4009740 tx=0x7fd2e401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/4282188679 shutdown_connections 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 --2- 172.21.15.96:0/4282188679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e8137110 0x7fd2e81374f0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/4282188679 >> 172.21.15.96:0/4282188679 conn(0x7fd2e806e020 msgr2=0x7fd2e8070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:18.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/4282188679 shutdown_connections 2024-09-13T17:44:18.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/4282188679 wait complete. 2024-09-13T17:44:18.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 Processor -- start 2024-09-13T17:44:18.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- start start 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2e80a6100 con 0x7fd2e809ba80 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ed2a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ed2a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55314/0 (socket says 172.21.15.96:55314) 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ed2a1640 1 -- 172.21.15.96:0/2639048564 learned_addr learned my addr 172.21.15.96:0/2639048564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:18.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ed2a1640 1 -- 172.21.15.96:0/2639048564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e4009420 con 0x7fd2e809ba80 2024-09-13T17:44:18.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ed2a1640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4019fc0 tx=0x7fd2e401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:18.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e4004160 con 0x7fd2e809ba80 2024-09-13T17:44:18.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e40042c0 con 0x7fd2e809ba80 2024-09-13T17:44:18.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2e80a6670 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd2e4003620 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2e80a5840 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2e809cf80 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fd2e40b2050 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 0x7fd2cc037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fd2e40e2250 con 0x7fd2e809ba80 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ecaa0640 1 -- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 msgr2=0x7fd2cc037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:18.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.932+0000 7fd2ecaa0640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 0x7fd2cc037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:18.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:18.940+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd2e40c7b00 con 0x7fd2e809ba80 2024-09-13T17:44:19.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.088+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2e8073a50 con 0x7fd2e809ba80 2024-09-13T17:44:19.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.088+0000 7fd2ddffb640 1 -- 172.21.15.96:0/2639048564 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2e40b00a0 con 0x7fd2e809ba80 2024-09-13T17:44:19.093 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 msgr2=0x7fd2cc037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 0x7fd2cc037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 msgr2=0x7fd2e811dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd2e4019fc0 tx=0x7fd2e401abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 shutdown_connections 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fd2cc035570 0x7fd2cc037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 --2- 172.21.15.96:0/2639048564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2e809ba80 0x7fd2e811dd70 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 >> 172.21.15.96:0/2639048564 conn(0x7fd2e806e020 msgr2=0x7fd2e8136100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 shutdown_connections 2024-09-13T17:44:19.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.092+0000 7fd2ef505640 1 -- 172.21.15.96:0/2639048564 wait complete. 2024-09-13T17:44:19.112 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:19.161 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.155+0000 7feb80efc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:44:19.213 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.207+0000 7feb80efc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:44:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.223+0000 7f6225555640 1 Processor -- start 2024-09-13T17:44:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.223+0000 7f6225555640 1 -- start start 2024-09-13T17:44:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f6225555640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f62201357b0 0x7f6220139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f6225555640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6220134af0 con 0x7f622009b0a0 2024-09-13T17:44:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f6225555640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f622009b480 con 0x7f62201357b0 2024-09-13T17:44:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f62201357b0 0x7f6220139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f62201357b0 0x7f6220139e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55328/0 (socket says 172.21.15.96:55328) 2024-09-13T17:44:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f621effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f622009b0a0 0x7f62201350a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38890/0 (socket says 172.21.15.96:38890) 2024-09-13T17:44:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 -- 172.21.15.96:0/290608136 learned_addr learned my addr 172.21.15.96:0/290608136 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 -- 172.21.15.96:0/290608136 >> v1:172.21.15.96:6789/0 conn(0x7f622009b0a0 legacy=0x7f62201350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 -- 172.21.15.96:0/290608136 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622009ccc0 con 0x7f62201357b0 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f62167fc640 1 --2- 172.21.15.96:0/290608136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62201357b0 0x7f6220139e70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62080097f0 tx=0x7f620801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53eb3fdca3afa7ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f621e7fc640 1 -- 172.21.15.96:0/290608136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62080ac540 con 0x7f62201357b0 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f621e7fc640 1 -- 172.21.15.96:0/290608136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62080ac6a0 con 0x7f62201357b0 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.227+0000 7f621e7fc640 1 -- 172.21.15.96:0/290608136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f62080ac960 con 0x7f62201357b0 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/290608136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62201357b0 msgr2=0x7f6220139e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 --2- 172.21.15.96:0/290608136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62201357b0 0x7f6220139e70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f62080097f0 tx=0x7f620801c3f0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/290608136 shutdown_connections 2024-09-13T17:44:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 --2- 172.21.15.96:0/290608136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62201357b0 0x7f6220139e70 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/290608136 >> 172.21.15.96:0/290608136 conn(0x7f622006e020 msgr2=0x7f6220070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/290608136 shutdown_connections 2024-09-13T17:44:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/290608136 wait complete. 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 Processor -- start 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- start start 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62200a00d0 con 0x7f622009b0a0 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f621effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f621effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55332/0 (socket says 172.21.15.96:55332) 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f621effd640 1 -- 172.21.15.96:0/932716876 learned_addr learned my addr 172.21.15.96:0/932716876 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f621effd640 1 -- 172.21.15.96:0/932716876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62080094d0 con 0x7f622009b0a0 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f621effd640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f621000c700 tx=0x7f621000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6210005db0 con 0x7f622009b0a0 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621000a660 con 0x7f622009b0a0 2024-09-13T17:44:19.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622009e930 con 0x7f622009b0a0 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f621000a900 con 0x7f622009b0a0 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6220133880 con 0x7f622009b0a0 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62201350a0 con 0x7f622009b0a0 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f621000aa60 con 0x7f622009b0a0 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 0x7f6200037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f62100474a0 con 0x7f622009b0a0 2024-09-13T17:44:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f62167fc640 1 -- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 msgr2=0x7f6200037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.231+0000 7f62167fc640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 0x7f6200037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.239+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6210007070 con 0x7f622009b0a0 2024-09-13T17:44:19.265 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.259+0000 7feb80efc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:44:19.341 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.335+0000 7feb80efc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:44:19.390 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.383+0000 7feb80efc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:44:19.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6220073a50 con 0x7f622009b0a0 2024-09-13T17:44:19.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6217fff640 1 -- 172.21.15.96:0/932716876 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6210017020 con 0x7f622009b0a0 2024-09-13T17:44:19.401 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 msgr2=0x7f6200037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6225555640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 0x7f6200037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 msgr2=0x7f6220138560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.395+0000 7f6225555640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f621000c700 tx=0x7f621000cbc0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 shutdown_connections 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6200035570 0x7f6200037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 --2- 172.21.15.96:0/932716876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f622009b0a0 0x7f6220138560 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 >> 172.21.15.96:0/932716876 conn(0x7f622006e020 msgr2=0x7f62200991b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 shutdown_connections 2024-09-13T17:44:19.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.399+0000 7f6225555640 1 -- 172.21.15.96:0/932716876 wait complete. 2024-09-13T17:44:19.418 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:19.467 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.463+0000 7feb80efc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:44:19.517 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.511+0000 7feb80efc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:44:19.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68fd882640 1 Processor -- start 2024-09-13T17:44:19.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68fd882640 1 -- start start 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68fd882640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f68f813af50 0x7f68f813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68fd882640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f8138050 con 0x7f68f809d690 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68fd882640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f809c410 con 0x7f68f813af50 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68f67fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f68f813af50 0x7f68f813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68f6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f68f809d690 0x7f68f8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38894/0 (socket says 172.21.15.96:38894) 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.539+0000 7f68f6ffd640 1 -- 172.21.15.96:0/806636193 learned_addr learned my addr 172.21.15.96:0/806636193 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f67fc640 1 -- 172.21.15.96:0/806636193 >> v1:172.21.15.96:6789/0 conn(0x7f68f809d690 legacy=0x7f68f8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f5ffb640 1 -- 172.21.15.96:0/806636193 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046954968 0 0) 0x7f68e8001120 con 0x7f68f809d690 2024-09-13T17:44:19.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f67fc640 1 -- 172.21.15.96:0/806636193 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f809cba0 con 0x7f68f813af50 2024-09-13T17:44:19.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f67fc640 1 --2- 172.21.15.96:0/806636193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f813af50 0x7f68f813d340 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009740 tx=0x7f68e001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2755777a1f94a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f5ffb640 1 -- 172.21.15.96:0/806636193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68e00a8840 con 0x7f68f813af50 2024-09-13T17:44:19.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f5ffb640 1 -- 172.21.15.96:0/806636193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e00a89a0 con 0x7f68f813af50 2024-09-13T17:44:19.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68f5ffb640 1 -- 172.21.15.96:0/806636193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68e00a8c60 con 0x7f68f813af50 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 -- 172.21.15.96:0/806636193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f813af50 msgr2=0x7f68f813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 --2- 172.21.15.96:0/806636193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f813af50 0x7f68f813d340 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009740 tx=0x7f68e001ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 -- 172.21.15.96:0/806636193 shutdown_connections 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 --2- 172.21.15.96:0/806636193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f813af50 0x7f68f813d340 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 -- 172.21.15.96:0/806636193 >> 172.21.15.96:0/806636193 conn(0x7f68f806bc60 msgr2=0x7f68f806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 -- 172.21.15.96:0/806636193 shutdown_connections 2024-09-13T17:44:19.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 -- 172.21.15.96:0/806636193 wait complete. 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.543+0000 7f68fd882640 1 Processor -- start 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 -- start start 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80a5c90 con 0x7f68f809d690 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55346/0 (socket says 172.21.15.96:55346) 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f6ffd640 1 -- 172.21.15.96:0/3894691339 learned_addr learned my addr 172.21.15.96:0/3894691339 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f6ffd640 1 -- 172.21.15.96:0/3894691339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e0009420 con 0x7f68f809d690 2024-09-13T17:44:19.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f6ffd640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c390 tx=0x7f68e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68e8007410 con 0x7f68f809d690 2024-09-13T17:44:19.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e8007590 con 0x7f68f809d690 2024-09-13T17:44:19.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f68e80047f0 con 0x7f68f809d690 2024-09-13T17:44:19.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f80a6f20 con 0x7f68f809d690 2024-09-13T17:44:19.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f8137400 con 0x7f68f809d690 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f809bc00 con 0x7f68f809d690 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f68e8004950 con 0x7f68f809d690 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 0x7f68d8037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f68e8047d70 con 0x7f68f809d690 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f67fc640 1 -- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 msgr2=0x7f68d8037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:19.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.547+0000 7f68f67fc640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 0x7f68d8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:19.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.551+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f68e801b520 con 0x7f68f809d690 2024-09-13T17:44:19.662 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.655+0000 7feb80efc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:44:19.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.703+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68f8138050 con 0x7f68f809d690 2024-09-13T17:44:19.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.703+0000 7f68effff640 1 -- 172.21.15.96:0/3894691339 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v42) v1 ==== 106+0+11 (secure 0 0 0) 0x7f68f8138050 con 0x7f68f809d690 2024-09-13T17:44:19.709 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:19.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 msgr2=0x7f68d8037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:19.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 0x7f68d8037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 msgr2=0x7f68f80a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c390 tx=0x7f68e800c850 comp rx=0 tx=0).stop 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 shutdown_connections 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f68d8035570 0x7f68d8037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 --2- 172.21.15.96:0/3894691339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68f809d690 0x7f68f80a69e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 >> 172.21.15.96:0/3894691339 conn(0x7f68f806bc60 msgr2=0x7f68f809f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 shutdown_connections 2024-09-13T17:44:19.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.707+0000 7f68fd882640 1 -- 172.21.15.96:0/3894691339 wait complete. 2024-09-13T17:44:19.717 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.711+0000 7feb80efc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:44:19.727 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:19.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f878b811640 1 Processor -- start 2024-09-13T17:44:19.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f878b811640 1 -- start start 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f878b811640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87840a1f60 0x7f878413d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f878b811640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87840a2cb0 con 0x7f87840a1470 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f878b811640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f878409de30 con 0x7f87840a1f60 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f8788dac640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87840a1f60 0x7f878413d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f87895ad640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f87840a1470 0x7f87840a1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38908/0 (socket says 172.21.15.96:38908) 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f87895ad640 1 -- 172.21.15.96:0/2570857280 learned_addr learned my addr 172.21.15.96:0/2570857280 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:19.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f8788dac640 1 -- 172.21.15.96:0/2570857280 >> v1:172.21.15.96:6789/0 conn(0x7f87840a1470 legacy=0x7f87840a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.831+0000 7f8788dac640 1 -- 172.21.15.96:0/2570857280 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878409db90 con 0x7f87840a1f60 2024-09-13T17:44:19.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.835+0000 7f8788dac640 1 --2- 172.21.15.96:0/2570857280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1f60 0x7f878413d2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8780009740 tx=0x7f878001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71074430e4008559 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f877bfff640 1 -- 172.21.15.96:0/2570857280 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87800ab530 con 0x7f87840a1f60 2024-09-13T17:44:19.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f877bfff640 1 -- 172.21.15.96:0/2570857280 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87800a7050 con 0x7f87840a1f60 2024-09-13T17:44:19.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f877bfff640 1 -- 172.21.15.96:0/2570857280 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87800ab530 con 0x7f87840a1f60 2024-09-13T17:44:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- 172.21.15.96:0/2570857280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1f60 msgr2=0x7f878413d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 --2- 172.21.15.96:0/2570857280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1f60 0x7f878413d2a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8780009740 tx=0x7f878001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- 172.21.15.96:0/2570857280 shutdown_connections 2024-09-13T17:44:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 --2- 172.21.15.96:0/2570857280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1f60 0x7f878413d2a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:19.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- 172.21.15.96:0/2570857280 >> 172.21.15.96:0/2570857280 conn(0x7f878406a150 msgr2=0x7f878406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:19.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- 172.21.15.96:0/2570857280 shutdown_connections 2024-09-13T17:44:19.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- 172.21.15.96:0/2570857280 wait complete. 2024-09-13T17:44:19.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 Processor -- start 2024-09-13T17:44:19.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.839+0000 7f878b811640 1 -- start start 2024-09-13T17:44:19.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f878b811640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f878b811640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f878409cd30 con 0x7f87840a1470 2024-09-13T17:44:19.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f87895ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:19.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f87895ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55366/0 (socket says 172.21.15.96:55366) 2024-09-13T17:44:19.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f87895ad640 1 -- 172.21.15.96:0/1703753916 learned_addr learned my addr 172.21.15.96:0/1703753916 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:19.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f87895ad640 1 -- 172.21.15.96:0/1703753916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8780009420 con 0x7f87840a1470 2024-09-13T17:44:19.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f87895ad640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f877400c3a0 tx=0x7f877400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:19.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8774005c20 con 0x7f87840a1470 2024-09-13T17:44:19.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f878409a610 con 0x7f87840a1470 2024-09-13T17:44:19.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f878409a290 con 0x7f87840a1470 2024-09-13T17:44:19.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8774005d80 con 0x7f87840a1470 2024-09-13T17:44:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.843+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8774007200 con 0x7f87840a1470 2024-09-13T17:44:19.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.847+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8764005260 con 0x7f87840a1470 2024-09-13T17:44:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.847+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f8774007360 con 0x7f87840a1470 2024-09-13T17:44:19.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.847+0000 7f8779ffb640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 0x7f8768037a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.855+0000 7f8788dac640 1 -- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 msgr2=0x7f8768037a80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.855+0000 7f8788dac640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 0x7f8768037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.855+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f87740104c0 con 0x7f87840a1470 2024-09-13T17:44:19.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:19.859+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8774007610 con 0x7f87840a1470 2024-09-13T17:44:19.937 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:44:19.937 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:44:19.937 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:44:19.944 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:19.939+0000 7feb80efc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:44:20.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7f8764005a90 con 0x7f87840a1470 2024-09-13T17:44:20.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f8779ffb640 1 -- 172.21.15.96:0/1703753916 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v42) v1 ==== 102+0+11 (secure 0 0 0) 0x7f8774017030 con 0x7f87840a1470 2024-09-13T17:44:20.060 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 msgr2=0x7f8768037a80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 0x7f8768037a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 msgr2=0x7f878409afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f877400c3a0 tx=0x7f877400c860 comp rx=0 tx=0).stop 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 shutdown_connections 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f87680355c0 0x7f8768037a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 --2- 172.21.15.96:0/1703753916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87840a1470 0x7f878409afe0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 >> 172.21.15.96:0/1703753916 conn(0x7f878406a150 msgr2=0x7f878406e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 shutdown_connections 2024-09-13T17:44:20.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.055+0000 7f878b811640 1 -- 172.21.15.96:0/1703753916 wait complete. 2024-09-13T17:44:20.078 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:44:20.086 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.079+0000 7feb80efc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:44:20.198 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.191+0000 7feb80efc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:44:20.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6127f1b640 1 Processor -- start 2024-09-13T17:44:20.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6127f1b640 1 -- start start 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6127f1b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f612009a0a0 0x7f612013af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6127f1b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6120135cd0 con 0x7f6120136170 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6127f1b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6120099c90 con 0x7f612009a0a0 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f61254b6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f612009a0a0 0x7f612013af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6125cb7640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6120136170 0x7f6120136550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38912/0 (socket says 172.21.15.96:38912) 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6125cb7640 1 -- 172.21.15.96:0/2460509937 learned_addr learned my addr 172.21.15.96:0/2460509937 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f61254b6640 1 -- 172.21.15.96:0/2460509937 >> v1:172.21.15.96:6789/0 conn(0x7f6120136170 legacy=0x7f6120136550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6124cb5640 1 -- 172.21.15.96:0/2460509937 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765592948 0 0) 0x7f6120135cd0 con 0x7f6120136170 2024-09-13T17:44:20.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f61254b6640 1 -- 172.21.15.96:0/2460509937 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612009d980 con 0x7f612009a0a0 2024-09-13T17:44:20.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f61254b6640 1 --2- 172.21.15.96:0/2460509937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612013af20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6110009740 tx=0x7f611001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76160670e7ab7109 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6124cb5640 1 -- 172.21.15.96:0/2460509937 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61100ab530 con 0x7f612009a0a0 2024-09-13T17:44:20.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6124cb5640 1 -- 172.21.15.96:0/2460509937 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61100a7050 con 0x7f612009a0a0 2024-09-13T17:44:20.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.195+0000 7f6124cb5640 1 -- 172.21.15.96:0/2460509937 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61100ab7e0 con 0x7f612009a0a0 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/2460509937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 msgr2=0x7f612013af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 --2- 172.21.15.96:0/2460509937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612013af20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6110009740 tx=0x7f611001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/2460509937 shutdown_connections 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 --2- 172.21.15.96:0/2460509937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612013af20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/2460509937 >> 172.21.15.96:0/2460509937 conn(0x7f612009c210 msgr2=0x7f6120071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/2460509937 shutdown_connections 2024-09-13T17:44:20.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/2460509937 wait complete. 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 Processor -- start 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- start start 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61200a8e90 con 0x7f612009a0a0 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6125cb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6125cb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55376/0 (socket says 172.21.15.96:55376) 2024-09-13T17:44:20.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6125cb7640 1 -- 172.21.15.96:0/1153114092 learned_addr learned my addr 172.21.15.96:0/1153114092 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6125cb7640 1 -- 172.21.15.96:0/1153114092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6110009420 con 0x7f612009a0a0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6125cb7640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c840 tx=0x7f611c00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f611c00a660 con 0x7f612009a0a0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c00a7e0 con 0x7f612009a0a0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f611c00aac0 con 0x7f612009a0a0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/1153114092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6120139870 con 0x7f612009a0a0 2024-09-13T17:44:20.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6127f1b640 1 -- 172.21.15.96:0/1153114092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6120139b10 con 0x7f612009a0a0 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f611c00ac20 con 0x7f612009a0a0 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc005260 con 0x7f612009a0a0 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 0x7f6104037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61254b6640 1 -- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 msgr2=0x7f6104037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f611c0476b0 con 0x7f612009a0a0 2024-09-13T17:44:20.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.199+0000 7f61254b6640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 0x7f6104037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:20.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.207+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f611c01a6b0 con 0x7f612009a0a0 2024-09-13T17:44:20.262 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.255+0000 7feb80efc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:44:20.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.363+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60fc005510 con 0x7f612009a0a0 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f61167fc640 1 -- 172.21.15.96:0/1153114092 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v42) v1 ==== 80+0+467 (secure 0 0 0) 0x7f611c018020 con 0x7f612009a0a0 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2785343960 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3263021005 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2962687900 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/715245426 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3193678716 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.371 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3614824494 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:20.372 INFO:teuthology.orchestra.run.smithi096.stderr:listed 8 entries 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 msgr2=0x7f6104037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 0x7f6104037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 msgr2=0x7f612006b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f611c00c840 tx=0x7f611c00cd00 comp rx=0 tx=0).stop 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 shutdown_connections 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f6104035570 0x7f6104037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 --2- 172.21.15.96:0/1153114092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f612009a0a0 0x7f612006b090 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 >> 172.21.15.96:0/1153114092 conn(0x7f612009c210 msgr2=0x7f6120132e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 shutdown_connections 2024-09-13T17:44:20.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.367+0000 7f6103fff640 1 -- 172.21.15.96:0/1153114092 wait complete. 2024-09-13T17:44:20.389 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:44:20.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:20.390 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:20.397 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.391+0000 7feb80efc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:44:20.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[38269]: fuse finished with error 0 and tester_r 0 2024-09-13T17:44:20.406 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:44:20.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:20.407 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:44:20.572 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:20.573 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:20.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:20.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5cbb8640 1 Processor -- start 2024-09-13T17:44:20.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5cbb8640 1 -- start start 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5cbb8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcd580a1f60 0x7fcd5813d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5cbb8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580a2cb0 con 0x7fcd580a1470 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5cbb8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd5809de30 con 0x7fcd580a1f60 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd55d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcd580a1f60 0x7fcd5813d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcd580a1470 0x7fcd580a1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38924/0 (socket says 172.21.15.96:38924) 2024-09-13T17:44:20.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5659c640 1 -- 172.21.15.96:0/4107653957 learned_addr learned my addr 172.21.15.96:0/4107653957 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:20.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd55d9b640 1 -- 172.21.15.96:0/4107653957 >> v1:172.21.15.96:6789/0 conn(0x7fcd580a1470 legacy=0x7fcd580a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd55d9b640 1 -- 172.21.15.96:0/4107653957 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd5809db90 con 0x7fcd580a1f60 2024-09-13T17:44:20.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd55d9b640 1 --2- 172.21.15.96:0/4107653957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1f60 0x7fcd5813d2a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009740 tx=0x7fcd4001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cfdc9a77fc2d81 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5559a640 1 -- 172.21.15.96:0/4107653957 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd400ab530 con 0x7fcd580a1f60 2024-09-13T17:44:20.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5559a640 1 -- 172.21.15.96:0/4107653957 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd400a7050 con 0x7fcd580a1f60 2024-09-13T17:44:20.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.775+0000 7fcd5559a640 1 -- 172.21.15.96:0/4107653957 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd400ab7e0 con 0x7fcd580a1f60 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/4107653957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1f60 msgr2=0x7fcd5813d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/4107653957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1f60 0x7fcd5813d2a0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009740 tx=0x7fcd4001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/4107653957 shutdown_connections 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/4107653957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1f60 0x7fcd5813d2a0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/4107653957 >> 172.21.15.96:0/4107653957 conn(0x7fcd5806a150 msgr2=0x7fcd5806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/4107653957 shutdown_connections 2024-09-13T17:44:20.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/4107653957 wait complete. 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 Processor -- start 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- start start 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58120c90 con 0x7fcd580a1470 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55388/0 (socket says 172.21.15.96:55388) 2024-09-13T17:44:20.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5659c640 1 -- 172.21.15.96:0/3387492894 learned_addr learned my addr 172.21.15.96:0/3387492894 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5659c640 1 -- 172.21.15.96:0/3387492894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40009420 con 0x7fcd580a1470 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5659c640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c480 tx=0x7fcd4800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd48005c20 con 0x7fcd580a1470 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48005d80 con 0x7fcd580a1470 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd58122bc0 con 0x7fcd580a1470 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd480071a0 con 0x7fcd580a1470 2024-09-13T17:44:20.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd58120ec0 con 0x7fcd580a1470 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fcd48007300 con 0x7fcd580a1470 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58121930 con 0x7fcd580a1470 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 0x7fcd38037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fcd48047080 con 0x7fcd580a1470 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd55d9b640 1 -- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 msgr2=0x7fcd38037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:20.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.779+0000 7fcd55d9b640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 0x7fcd38037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:20.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.787+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcd48010300 con 0x7fcd580a1470 2024-09-13T17:44:20.884 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.879+0000 7feb80efc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:44:20.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.943+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd580a2cb0 con 0x7fcd580a1470 2024-09-13T17:44:20.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd4effd640 1 -- 172.21.15.96:0/3387492894 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+14765 (secure 0 0 0) 0x7fcd580a2cb0 con 0x7fcd580a1470 2024-09-13T17:44:20.952 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:20.952 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":42,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:43:57.009683+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:20.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 msgr2=0x7fcd38037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:20.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 0x7fcd38037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 msgr2=0x7fcd58122680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fcd4800c480 tx=0x7fcd4800c940 comp rx=0 tx=0).stop 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 shutdown_connections 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fcd38035570 0x7fcd38037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 --2- 172.21.15.96:0/3387492894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd580a1470 0x7fcd58122680 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 >> 172.21.15.96:0/3387492894 conn(0x7fcd5806a150 msgr2=0x7fcd5813cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 shutdown_connections 2024-09-13T17:44:20.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:20.947+0000 7fcd5cbb8640 1 -- 172.21.15.96:0/3387492894 wait complete. 2024-09-13T17:44:20.958 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:20.955+0000 7feb80efc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:44:20.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:20.971 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:21.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb11a145640 1 Processor -- start 2024-09-13T17:44:21.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb11a145640 1 -- start start 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb11a145640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb11409d690 0x7fb114138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb11a145640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb114138050 con 0x7fb1141387d0 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb11a145640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11409c410 con 0x7fb11409d690 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb1137fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb11409d690 0x7fb114138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb112ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb1141387d0 0x7fb114134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38926/0 (socket says 172.21.15.96:38926) 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb112ffd640 1 -- 172.21.15.96:0/3036884619 learned_addr learned my addr 172.21.15.96:0/3036884619 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb1137fe640 1 -- 172.21.15.96:0/3036884619 >> v1:172.21.15.96:6789/0 conn(0x7fb1141387d0 legacy=0x7fb114134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.075+0000 7fb1137fe640 1 -- 172.21.15.96:0/3036884619 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11409cba0 con 0x7fb11409d690 2024-09-13T17:44:21.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1137fe640 1 --2- 172.21.15.96:0/3036884619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb114138290 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb104009740 tx=0x7fb10401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de49977469c9d9ad server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1127fc640 1 -- 172.21.15.96:0/3036884619 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1040a8840 con 0x7fb11409d690 2024-09-13T17:44:21.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1127fc640 1 -- 172.21.15.96:0/3036884619 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1040a89a0 con 0x7fb11409d690 2024-09-13T17:44:21.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1127fc640 1 -- 172.21.15.96:0/3036884619 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1040a8c60 con 0x7fb11409d690 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- 172.21.15.96:0/3036884619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 msgr2=0x7fb114138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 --2- 172.21.15.96:0/3036884619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb114138290 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fb104009740 tx=0x7fb10401ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- 172.21.15.96:0/3036884619 shutdown_connections 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 --2- 172.21.15.96:0/3036884619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb114138290 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- 172.21.15.96:0/3036884619 >> 172.21.15.96:0/3036884619 conn(0x7fb11406bc60 msgr2=0x7fb11406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- 172.21.15.96:0/3036884619 shutdown_connections 2024-09-13T17:44:21.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- 172.21.15.96:0/3036884619 wait complete. 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 Processor -- start 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- start start 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb11a145640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1140a4d00 con 0x7fb11409d690 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55414/0 (socket says 172.21.15.96:55414) 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1137fe640 1 -- 172.21.15.96:0/1448597752 learned_addr learned my addr 172.21.15.96:0/1448597752 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.079+0000 7fb1137fe640 1 -- 172.21.15.96:0/1448597752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009420 con 0x7fb11409d690 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb1137fe640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb1040b3040 tx=0x7fb10401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1040b0800 con 0x7fb11409d690 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1040b0960 con 0x7fb11409d690 2024-09-13T17:44:21.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1140a4f30 con 0x7fb11409d690 2024-09-13T17:44:21.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb1040b0c00 con 0x7fb11409d690 2024-09-13T17:44:21.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1140a6f00 con 0x7fb11409d690 2024-09-13T17:44:21.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fb1040b7040 con 0x7fb11409d690 2024-09-13T17:44:21.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1140a4420 con 0x7fb11409d690 2024-09-13T17:44:21.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 0x7fb100037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb112ffd640 1 -- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 msgr2=0x7fb100037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:21.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7fb1040e5b20 con 0x7fb11409d690 2024-09-13T17:44:21.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.083+0000 7fb112ffd640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 0x7fb100037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:21.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.091+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb1040af3a0 con 0x7fb11409d690 2024-09-13T17:44:21.112 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.107+0000 7feb80efc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:44:21.186 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.179+0000 7feb80efc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:44:21.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb11409da70 con 0x7fb11409d690 2024-09-13T17:44:21.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb0fbfff640 1 -- 172.21.15.96:0/1448597752 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1040b5070 con 0x7fb11409d690 2024-09-13T17:44:21.260 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 msgr2=0x7fb100037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 0x7fb100037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 msgr2=0x7fb1140a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fb1040b3040 tx=0x7fb10401b9b0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 shutdown_connections 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7fb100035570 0x7fb100037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 --2- 172.21.15.96:0/1448597752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb11409d690 0x7fb1140a69c0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 >> 172.21.15.96:0/1448597752 conn(0x7fb11406bc60 msgr2=0x7fb1140993a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.255+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 shutdown_connections 2024-09-13T17:44:21.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.259+0000 7fb11a145640 1 -- 172.21.15.96:0/1448597752 wait complete. 2024-09-13T17:44:21.279 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:21.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1453ead640 1 Processor -- start 2024-09-13T17:44:21.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1453ead640 1 -- start start 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1453ead640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f144c09b0a0 0x7f144c1350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1453ead640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f144c135a30 con 0x7f144c1355e0 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1453ead640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144c134af0 con 0x7f144c09b0a0 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1451c49640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f144c09b0a0 0x7f144c1350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1451448640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f144c1355e0 0x7f144c131540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38930/0 (socket says 172.21.15.96:38930) 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.399+0000 7f1451448640 1 -- 172.21.15.96:0/3893760167 learned_addr learned my addr 172.21.15.96:0/3893760167 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1451c49640 1 -- 172.21.15.96:0/3893760167 >> v1:172.21.15.96:6789/0 conn(0x7f144c1355e0 legacy=0x7f144c131540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1451c49640 1 -- 172.21.15.96:0/3893760167 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144c09b480 con 0x7f144c09b0a0 2024-09-13T17:44:21.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1451c49640 1 --2- 172.21.15.96:0/3893760167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c1350a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1444009740 tx=0x7f144401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b077d410d42ac21e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1450c47640 1 -- 172.21.15.96:0/3893760167 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f144c132460 con 0x7f144c09b0a0 2024-09-13T17:44:21.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1450c47640 1 -- 172.21.15.96:0/3893760167 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14440a7050 con 0x7f144c09b0a0 2024-09-13T17:44:21.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1450c47640 1 -- 172.21.15.96:0/3893760167 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14440042f0 con 0x7f144c09b0a0 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 -- 172.21.15.96:0/3893760167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 msgr2=0x7f144c1350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 --2- 172.21.15.96:0/3893760167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c1350a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1444009740 tx=0x7f144401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 -- 172.21.15.96:0/3893760167 shutdown_connections 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 --2- 172.21.15.96:0/3893760167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c1350a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 -- 172.21.15.96:0/3893760167 >> 172.21.15.96:0/3893760167 conn(0x7f144c06e020 msgr2=0x7f144c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 -- 172.21.15.96:0/3893760167 shutdown_connections 2024-09-13T17:44:21.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 -- 172.21.15.96:0/3893760167 wait complete. 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.403+0000 7f1453ead640 1 Processor -- start 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 -- start start 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c13aa60 con 0x7f144c09b0a0 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451c49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55428/0 (socket says 172.21.15.96:55428) 2024-09-13T17:44:21.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451c49640 1 -- 172.21.15.96:0/2614179617 learned_addr learned my addr 172.21.15.96:0/2614179617 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:21.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451c49640 1 -- 172.21.15.96:0/2614179617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444009420 con 0x7f144c09b0a0 2024-09-13T17:44:21.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451c49640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1444019fc0 tx=0x7f1444019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1444003aa0 con 0x7f144c09b0a0 2024-09-13T17:44:21.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1444003c00 con 0x7f144c09b0a0 2024-09-13T17:44:21.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c131420 con 0x7f144c09b0a0 2024-09-13T17:44:21.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f14440ab440 con 0x7f144c09b0a0 2024-09-13T17:44:21.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c134270 con 0x7f144c09b0a0 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 21) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f14440b2040 con 0x7f144c09b0a0 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c12ed70 con 0x7f144c09b0a0 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 0x7f1430037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4643+0+0 (secure 0 0 0) 0x7f14440e2060 con 0x7f144c09b0a0 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451448640 1 -- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 msgr2=0x7f1430037a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/1425974344 2024-09-13T17:44:21.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.407+0000 7f1451448640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 0x7f1430037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:44:21.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.411+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14440aa7b0 con 0x7f144c09b0a0 2024-09-13T17:44:21.493 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.487+0000 7feb80efc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:44:21.548 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.543+0000 7feb80efc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:44:21.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.575+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 22) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f14440abd50 con 0x7f144c09b0a0 2024-09-13T17:44:21.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.575+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 msgr2=0x7f1430037a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:44:21.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.575+0000 7f14427fc640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 0x7f1430037a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.591+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f144c12faf0 con 0x7f144c09b0a0 2024-09-13T17:44:21.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.591+0000 7f14427fc640 1 -- 172.21.15.96:0/2614179617 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14440b0070 con 0x7f144c09b0a0 2024-09-13T17:44:21.598 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 msgr2=0x7f144c139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f1444019fc0 tx=0x7f1444019130 comp rx=0 tx=0).stop 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 shutdown_connections 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:6824/1425974344,v1:172.21.15.96:6825/1425974344] conn(0x7f1430035570 0x7f1430037a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 --2- 172.21.15.96:0/2614179617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144c09b0a0 0x7f144c139590 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 >> 172.21.15.96:0/2614179617 conn(0x7f144c06e020 msgr2=0x7f144c12dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 shutdown_connections 2024-09-13T17:44:21.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.595+0000 7f1453ead640 1 -- 172.21.15.96:0/2614179617 wait complete. 2024-09-13T17:44:21.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:21.646 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.646 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.646 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.646 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.647 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.639+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.659 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.659 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.659 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.659 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.659 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb0c379640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.660 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.660 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.660 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.660 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.660 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:44:21.655+0000 7feb09373640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:44:21.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e7a06640 1 Processor -- start 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e7a06640 1 -- start start 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e7a06640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55e0134e90 0x7f55e0071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e7a06640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e0070de0 con 0x7f55e0135270 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e7a06640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55e0071d60 con 0x7f55e0134e90 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e57a2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55e0134e90 0x7f55e0071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e4fa1640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f55e0135270 0x7f55e006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38932/0 (socket says 172.21.15.96:38932) 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e57a2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f55e0134e90 0x7f55e0071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55560/0 (socket says 172.21.15.96:55560) 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e4fa1640 1 -- 172.21.15.96:0/3355840463 learned_addr learned my addr 172.21.15.96:0/3355840463 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e57a2640 1 -- 172.21.15.96:0/3355840463 >> v1:172.21.15.96:6789/0 conn(0x7f55e0135270 legacy=0x7f55e006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.723+0000 7f55e57a2640 1 -- 172.21.15.96:0/3355840463 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e006f110 con 0x7f55e0134e90 2024-09-13T17:44:21.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 --2- 172.21.15.96:0/3355840463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e0071530 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009740 tx=0x7f55dc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12139df343d2ae server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d7fff640 1 -- 172.21.15.96:0/3355840463 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55dc0ab530 con 0x7f55e0134e90 2024-09-13T17:44:21.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d7fff640 1 -- 172.21.15.96:0/3355840463 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc0a7050 con 0x7f55e0134e90 2024-09-13T17:44:21.732 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d7fff640 1 -- 172.21.15.96:0/3355840463 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55dc0ab7e0 con 0x7f55e0134e90 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/3355840463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 msgr2=0x7f55e0071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 --2- 172.21.15.96:0/3355840463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e0071530 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f55dc009740 tx=0x7f55dc01c390 comp rx=0 tx=0).stop 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/3355840463 shutdown_connections 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 --2- 172.21.15.96:0/3355840463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e0071530 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/3355840463 >> 172.21.15.96:0/3355840463 conn(0x7f55e0098a60 msgr2=0x7f55e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/3355840463 shutdown_connections 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/3355840463 wait complete. 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 Processor -- start 2024-09-13T17:44:21.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- start start 2024-09-13T17:44:21.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e013cb00 con 0x7f55e0134e90 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55564/0 (socket says 172.21.15.96:55564) 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 -- 172.21.15.96:0/2128171823 learned_addr learned my addr 172.21.15.96:0/2128171823 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 -- 172.21.15.96:0/2128171823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc009420 con 0x7f55e0134e90 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e57a2640 1 --2- 172.21.15.96:0/2128171823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f55dc019fc0 tx=0x7f55dc01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:21.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55dc0b7450 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc0b75b0 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e00a1700 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55dc0b7870 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e00a6590 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e0071530 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.727+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f55dc0b79d0 con 0x7f55e0134e90 2024-09-13T17:44:21.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.731+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f55dc0e2210 con 0x7f55e0134e90 2024-09-13T17:44:21.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.735+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f55dc0c76c0 con 0x7f55e0134e90 2024-09-13T17:44:21.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55e00a2310 con 0x7f55e0134e90 2024-09-13T17:44:21.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55d5ffb640 1 -- 172.21.15.96:0/2128171823 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55dc0b00a0 con 0x7f55e0134e90 2024-09-13T17:44:21.884 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 msgr2=0x7f55e01d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 --2- 172.21.15.96:0/2128171823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f55dc019fc0 tx=0x7f55dc01abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 shutdown_connections 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 --2- 172.21.15.96:0/2128171823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55e0134e90 0x7f55e01d0360 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 >> 172.21.15.96:0/2128171823 conn(0x7f55e0098a60 msgr2=0x7f55e0133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 shutdown_connections 2024-09-13T17:44:21.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:21.879+0000 7f55e7a06640 1 -- 172.21.15.96:0/2128171823 wait complete. 2024-09-13T17:44:21.901 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:22.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ec730640 1 Processor -- start 2024-09-13T17:44:22.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ec730640 1 -- start start 2024-09-13T17:44:22.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ec730640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f41e409e820 0x7f41e413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ec730640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e40a68b0 con 0x7f41e4134e70 2024-09-13T17:44:22.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ec730640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e409ddc0 con 0x7f41e409e820 2024-09-13T17:44:22.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41e9ccb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f41e409e820 0x7f41e413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41e9ccb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f41e409e820 0x7f41e413ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55572/0 (socket says 172.21.15.96:55572) 2024-09-13T17:44:22.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41e9ccb640 1 -- 172.21.15.96:0/1723015474 learned_addr learned my addr 172.21.15.96:0/1723015474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:22.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41ea4cc640 1 --1- 172.21.15.96:0/1723015474 >> v1:172.21.15.96:6789/0 conn(0x7f41e4134e70 0x7f41e4135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:38934/0 (socket says 172.21.15.96:38934) 2024-09-13T17:44:22.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41e9ccb640 1 -- 172.21.15.96:0/1723015474 >> v1:172.21.15.96:6789/0 conn(0x7f41e4134e70 legacy=0x7f41e4135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.059+0000 7f41e9ccb640 1 -- 172.21.15.96:0/1723015474 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e4136970 con 0x7f41e409e820 2024-09-13T17:44:22.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41e9ccb640 1 --2- 172.21.15.96:0/1723015474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e413ae50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41d40097f0 tx=0x7f41d401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a88fb5124346b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41e94ca640 1 -- 172.21.15.96:0/1723015474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41d40ac540 con 0x7f41e409e820 2024-09-13T17:44:22.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41e94ca640 1 -- 172.21.15.96:0/1723015474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d40ac6a0 con 0x7f41e409e820 2024-09-13T17:44:22.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41e94ca640 1 -- 172.21.15.96:0/1723015474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41d40ac960 con 0x7f41e409e820 2024-09-13T17:44:22.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- 172.21.15.96:0/1723015474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 msgr2=0x7f41e413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 --2- 172.21.15.96:0/1723015474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e413ae50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f41d40097f0 tx=0x7f41d401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- 172.21.15.96:0/1723015474 shutdown_connections 2024-09-13T17:44:22.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 --2- 172.21.15.96:0/1723015474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e413ae50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- 172.21.15.96:0/1723015474 >> 172.21.15.96:0/1723015474 conn(0x7f41e4098a60 msgr2=0x7f41e409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- 172.21.15.96:0/1723015474 shutdown_connections 2024-09-13T17:44:22.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- 172.21.15.96:0/1723015474 wait complete. 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 Processor -- start 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- start start 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ec730640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e413a650 con 0x7f41e409e820 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ea4cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.063+0000 7f41ea4cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55578/0 (socket says 172.21.15.96:55578) 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ea4cc640 1 -- 172.21.15.96:0/3866165037 learned_addr learned my addr 172.21.15.96:0/3866165037 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ea4cc640 1 -- 172.21.15.96:0/3866165037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40094d0 con 0x7f41e409e820 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ea4cc640 1 --2- 172.21.15.96:0/3866165037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c7f0 tx=0x7f41dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41dc00a610 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc00a770 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e411c930 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f41dc00aa10 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e4071610 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f41dc00ab70 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e411cb30 con 0x7f41e409e820 2024-09-13T17:44:22.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.067+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f41dc046fd0 con 0x7f41e409e820 2024-09-13T17:44:22.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.071+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f41dc078db0 con 0x7f41e409e820 2024-09-13T17:44:22.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.227+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41e406a1b0 con 0x7f41e409e820 2024-09-13T17:44:22.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.227+0000 7f41daffd640 1 -- 172.21.15.96:0/3866165037 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v43) v1 ==== 106+0+11 (secure 0 0 0) 0x7f41dc017070 con 0x7f41e409e820 2024-09-13T17:44:22.234 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 msgr2=0x7f41e412ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 --2- 172.21.15.96:0/3866165037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00c7f0 tx=0x7f41dc00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 shutdown_connections 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 --2- 172.21.15.96:0/3866165037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41e409e820 0x7f41e412ef70 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 >> 172.21.15.96:0/3866165037 conn(0x7f41e4098a60 msgr2=0x7f41e4134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 shutdown_connections 2024-09-13T17:44:22.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.231+0000 7f41ec730640 1 -- 172.21.15.96:0/3866165037 wait complete. 2024-09-13T17:44:22.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:22.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.363+0000 7fd7d5c35640 1 Processor -- start 2024-09-13T17:44:22.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.363+0000 7fd7d5c35640 1 -- start start 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.363+0000 7fd7d5c35640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7d013af50 0x7fd7d013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.363+0000 7fd7d5c35640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d0138050 con 0x7fd7d009d690 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.363+0000 7fd7d5c35640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d009c410 con 0x7fd7d013af50 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7d013af50 0x7fd7d013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd7d013af50 0x7fd7d013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51916/0 (socket says 172.21.15.96:51916) 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 -- 172.21.15.96:0/4007756570 learned_addr learned my addr 172.21.15.96:0/4007756570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7cf7fe640 1 --1- 172.21.15.96:0/4007756570 >> v1:172.21.15.96:6789/0 conn(0x7fd7d009d690 0x7fd7d0138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45818/0 (socket says 172.21.15.96:45818) 2024-09-13T17:44:22.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 -- 172.21.15.96:0/4007756570 >> v1:172.21.15.96:6789/0 conn(0x7fd7d009d690 legacy=0x7fd7d0138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 -- 172.21.15.96:0/4007756570 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d009cba0 con 0x7fd7d013af50 2024-09-13T17:44:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ceffd640 1 --2- 172.21.15.96:0/4007756570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d013af50 0x7fd7d013d340 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40097f0 tx=0x7fd7c401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225a8178be6abcde server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ce7fc640 1 -- 172.21.15.96:0/4007756570 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7c40a8900 con 0x7fd7d013af50 2024-09-13T17:44:22.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ce7fc640 1 -- 172.21.15.96:0/4007756570 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c40a8a60 con 0x7fd7d013af50 2024-09-13T17:44:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7ce7fc640 1 -- 172.21.15.96:0/4007756570 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7c40a8d20 con 0x7fd7d013af50 2024-09-13T17:44:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 -- 172.21.15.96:0/4007756570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d013af50 msgr2=0x7fd7d013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 --2- 172.21.15.96:0/4007756570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d013af50 0x7fd7d013d340 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40097f0 tx=0x7fd7c401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 -- 172.21.15.96:0/4007756570 shutdown_connections 2024-09-13T17:44:22.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 --2- 172.21.15.96:0/4007756570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d013af50 0x7fd7d013d340 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 -- 172.21.15.96:0/4007756570 >> 172.21.15.96:0/4007756570 conn(0x7fd7d006bc60 msgr2=0x7fd7d006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 -- 172.21.15.96:0/4007756570 shutdown_connections 2024-09-13T17:44:22.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.367+0000 7fd7d5c35640 1 -- 172.21.15.96:0/4007756570 wait complete. 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 Processor -- start 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 -- start start 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00a4f90 con 0x7fd7d009d690 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51926/0 (socket says 172.21.15.96:51926) 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7cf7fe640 1 -- 172.21.15.96:0/2190500767 learned_addr learned my addr 172.21.15.96:0/2190500767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:22.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7cf7fe640 1 -- 172.21.15.96:0/2190500767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c40094d0 con 0x7fd7d009d690 2024-09-13T17:44:22.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7cf7fe640 1 --2- 172.21.15.96:0/2190500767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c3b0 tx=0x7fd7c000c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7c0007410 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c0007570 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00a2c20 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7c00047f0 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d01373b0 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) v1 ==== 43018+0+0 (secure 0 0 0) 0x7fd7c0004950 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d00a3f70 con 0x7fd7d009d690 2024-09-13T17:44:22.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.371+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fd7c0046ef0 con 0x7fd7d009d690 2024-09-13T17:44:22.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.375+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd7d00a3f70 con 0x7fd7d009d690 2024-09-13T17:44:22.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7fd7d0138050 con 0x7fd7d009d690 2024-09-13T17:44:22.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7b7fff640 1 -- 172.21.15.96:0/2190500767 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v43) v1 ==== 102+0+11 (secure 0 0 0) 0x7fd7d0138050 con 0x7fd7d009d690 2024-09-13T17:44:22.540 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:22.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 msgr2=0x7fd7d01c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 --2- 172.21.15.96:0/2190500767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c3b0 tx=0x7fd7c000c870 comp rx=0 tx=0).stop 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 shutdown_connections 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 --2- 172.21.15.96:0/2190500767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7d009d690 0x7fd7d01c9820 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.535+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 >> 172.21.15.96:0/2190500767 conn(0x7fd7d006bc60 msgr2=0x7fd7d0131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.539+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 shutdown_connections 2024-09-13T17:44:22.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.539+0000 7fd7d5c35640 1 -- 172.21.15.96:0/2190500767 wait complete. 2024-09-13T17:44:22.559 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:44:22.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa75e49f640 1 Processor -- start 2024-09-13T17:44:22.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa75e49f640 1 -- start start 2024-09-13T17:44:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa75e49f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa758137490 0x7fa758139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa75e49f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa758071670 con 0x7fa758134670 2024-09-13T17:44:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa75e49f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa758071360 con 0x7fa758137490 2024-09-13T17:44:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa7577fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa758137490 0x7fa758139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa757fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa758134670 0x7fa758134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45828/0 (socket says 172.21.15.96:45828) 2024-09-13T17:44:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa757fff640 1 -- 172.21.15.96:0/3190991842 learned_addr learned my addr 172.21.15.96:0/3190991842 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa7577fe640 1 -- 172.21.15.96:0/3190991842 >> v1:172.21.15.96:6789/0 conn(0x7fa758134670 legacy=0x7fa758134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa7577fe640 1 -- 172.21.15.96:0/3190991842 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7580a7200 con 0x7fa758137490 2024-09-13T17:44:22.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.679+0000 7fa756ffd640 1 -- 172.21.15.96:0/3190991842 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238185085 0 0) 0x7fa758071670 con 0x7fa758134670 2024-09-13T17:44:22.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa7577fe640 1 --2- 172.21.15.96:0/3190991842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758137490 0x7fa758139880 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009740 tx=0x7fa74c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fe18884a0546e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa756ffd640 1 -- 172.21.15.96:0/3190991842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa74c0ab530 con 0x7fa758137490 2024-09-13T17:44:22.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa756ffd640 1 -- 172.21.15.96:0/3190991842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0a7050 con 0x7fa758137490 2024-09-13T17:44:22.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa756ffd640 1 -- 172.21.15.96:0/3190991842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa74c0ab7e0 con 0x7fa758137490 2024-09-13T17:44:22.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3190991842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758137490 msgr2=0x7fa758139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3190991842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758137490 0x7fa758139880 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009740 tx=0x7fa74c01c390 comp rx=0 tx=0).stop 2024-09-13T17:44:22.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3190991842 shutdown_connections 2024-09-13T17:44:22.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3190991842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758137490 0x7fa758139880 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3190991842 >> 172.21.15.96:0/3190991842 conn(0x7fa758098a60 msgr2=0x7fa75809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3190991842 shutdown_connections 2024-09-13T17:44:22.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3190991842 wait complete. 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 Processor -- start 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- start start 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7580a2ef0 con 0x7fa758134670 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa757fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51934/0 (socket says 172.21.15.96:51934) 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa757fff640 1 -- 172.21.15.96:0/3879316270 learned_addr learned my addr 172.21.15.96:0/3879316270 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa757fff640 1 -- 172.21.15.96:0/3879316270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c009420 con 0x7fa758134670 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa757fff640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c420 tx=0x7fa74800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa7480074d0 con 0x7fa758134670 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748007630 con 0x7fa758134670 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7580a6880 con 0x7fa758134670 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa748004960 con 0x7fa758134670 2024-09-13T17:44:22.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.683+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7581324d0 con 0x7fa758134670 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fa748004ac0 con 0x7fa758134670 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7580a1f80 con 0x7fa758134670 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa754ff9640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 0x7fa744037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fa748046ff0 con 0x7fa758134670 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa7577fe640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 0x7fa744037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:22.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.687+0000 7fa7577fe640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 0x7fa744037760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:22.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.691+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa748078db0 con 0x7fa758134670 2024-09-13T17:44:22.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.847+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa7580a3040 con 0x7fa758134670 2024-09-13T17:44:22.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.847+0000 7fa754ff9640 1 -- 172.21.15.96:0/3879316270 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v43) v1 ==== 81+0+935 (secure 0 0 0) 0x7fa748018030 con 0x7fa758134670 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2758166216 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/1425974344 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1100369952 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/494405909 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3234139153 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/252612581 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2785343960 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3263021005 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2962687900 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2805890947 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.855 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/715245426 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.856 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/1425974344 2024-09-14T17:44:21.568899+0000 2024-09-13T17:44:22.856 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.856 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3193678716 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.856 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/526856805 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.856 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3614824494 2024-09-14T17:43:51.591620+0000 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:listed 16 entries 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 msgr2=0x7fa744037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 0x7fa744037760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0ad000 tx=0x7fa74c008040 comp rx=0 tx=0).stop 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 msgr2=0x7fa7580a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa74800c420 tx=0x7fa74800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 shutdown_connections 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa7440352a0 0x7fa744037760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 --2- 172.21.15.96:0/3879316270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa758134670 0x7fa7580a0850 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 >> 172.21.15.96:0/3879316270 conn(0x7fa758098a60 msgr2=0x7fa75806d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 shutdown_connections 2024-09-13T17:44:22.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:22.851+0000 7fa75e49f640 1 -- 172.21.15.96:0/3879316270 wait complete. 2024-09-13T17:44:22.871 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:44:22.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:22.871 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:22.882 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[40000]: fuse finished with error 0 and tester_r 0 2024-09-13T17:44:22.884 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:44:22.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:22.884 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:44:23.049 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:23.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c90d57640 1 Processor -- start 2024-09-13T17:44:23.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c90d57640 1 -- start start 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c90d57640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c8c134e70 0x7f5c8c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c90d57640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8c09ed60 con 0x7f5c8c135250 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c90d57640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c8c09ddc0 con 0x7f5c8c134e70 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c8c134e70 0x7f5c8c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c8c134e70 0x7f5c8c09e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51942/0 (socket says 172.21.15.96:51942) 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c89d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5c8c135250 0x7f5c8c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45844/0 (socket says 172.21.15.96:45844) 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 -- 172.21.15.96:0/2516689293 learned_addr learned my addr 172.21.15.96:0/2516689293 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:23.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 -- 172.21.15.96:0/2516689293 >> v1:172.21.15.96:6789/0 conn(0x7f5c8c135250 legacy=0x7f5c8c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 -- 172.21.15.96:0/2516689293 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c0a68b0 con 0x7f5c8c134e70 2024-09-13T17:44:23.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8a59c640 1 --2- 172.21.15.96:0/2516689293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c09e820 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0097f0 tx=0x7f5c7c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df24220d7de235c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8959a640 1 -- 172.21.15.96:0/2516689293 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c7c0ac540 con 0x7f5c8c134e70 2024-09-13T17:44:23.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8959a640 1 -- 172.21.15.96:0/2516689293 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0ac6a0 con 0x7f5c8c134e70 2024-09-13T17:44:23.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.207+0000 7f5c8959a640 1 -- 172.21.15.96:0/2516689293 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c7c0ac960 con 0x7f5c8c134e70 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/2516689293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 msgr2=0x7f5c8c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 --2- 172.21.15.96:0/2516689293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c09e820 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0097f0 tx=0x7f5c7c01c4e0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/2516689293 shutdown_connections 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 --2- 172.21.15.96:0/2516689293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c09e820 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/2516689293 >> 172.21.15.96:0/2516689293 conn(0x7f5c8c098a60 msgr2=0x7f5c8c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/2516689293 shutdown_connections 2024-09-13T17:44:23.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/2516689293 wait complete. 2024-09-13T17:44:23.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 Processor -- start 2024-09-13T17:44:23.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- start start 2024-09-13T17:44:23.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8c0a1710 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c8a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c8a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51956/0 (socket says 172.21.15.96:51956) 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c8a59c640 1 -- 172.21.15.96:0/3941146641 learned_addr learned my addr 172.21.15.96:0/3941146641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c8a59c640 1 -- 172.21.15.96:0/3941146641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0094d0 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c8a59c640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0190f0 tx=0x7f5c7c019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c7c0bb070 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0b7470 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8c0a3550 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c7c0b7730 con 0x7f5c8c134e70 2024-09-13T17:44:23.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8c0a4270 con 0x7f5c8c134e70 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f5c7c0b3070 con 0x7f5c8c134e70 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8c0a3350 con 0x7f5c8c134e70 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 0x7f5c6c037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f5c7c0e1ad0 con 0x7f5c8c134e70 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.211+0000 7f5c89d9b640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 0x7f5c6c037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.215+0000 7f5c89d9b640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 0x7f5c6c037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c74001600 tx=0x7f5c74007a50 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.219+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c7c0b4380 con 0x7f5c8c134e70 2024-09-13T17:44:23.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.407+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c8c1368b0 con 0x7f5c8c134e70 2024-09-13T17:44:23.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.407+0000 7f5c82ffd640 1 -- 172.21.15.96:0/3941146641 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4427 (secure 0 0 0) 0x7f5c7c0b1070 con 0x7f5c8c134e70 2024-09-13T17:44:23.414 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:23.414 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:23.415 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-09-13T17:44:23.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 msgr2=0x7f5c6c037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 0x7f5c6c037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5c74001600 tx=0x7f5c74007a50 comp rx=0 tx=0).stop 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 msgr2=0x7f5c8c12fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0190f0 tx=0x7f5c7c019280 comp rx=0 tx=0).stop 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 shutdown_connections 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5c6c0352a0 0x7f5c6c037760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 --2- 172.21.15.96:0/3941146641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c8c134e70 0x7f5c8c12fa60 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 >> 172.21.15.96:0/3941146641 conn(0x7f5c8c098a60 msgr2=0x7f5c8c133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 shutdown_connections 2024-09-13T17:44:23.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.411+0000 7f5c90d57640 1 -- 172.21.15.96:0/3941146641 wait complete. 2024-09-13T17:44:23.433 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fecedc15640 1 Processor -- start 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fecedc15640 1 -- start start 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fecedc15640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fece8134e70 0x7fece809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fecedc15640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece809ed60 con 0x7fece8135250 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fecedc15640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece809ddc0 con 0x7fece8134e70 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fece6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fece8135250 0x7fece8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45846/0 (socket says 172.21.15.96:45846) 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fece6ffd640 1 -- 172.21.15.96:0/509405930 learned_addr learned my addr 172.21.15.96:0/509405930 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:23.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fece77fe640 1 --2- 172.21.15.96:0/509405930 >> v2:172.21.15.96:3300/0 conn(0x7fece8134e70 0x7fece809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.535+0000 7fece67fc640 1 -- 172.21.15.96:0/509405930 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905227810 0 0) 0x7fecd0001120 con 0x7fece8135250 2024-09-13T17:44:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece67fc640 1 -- 172.21.15.96:0/509405930 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecd4003530 con 0x7fece8135250 2024-09-13T17:44:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece77fe640 1 -- 172.21.15.96:0/509405930 >> v1:172.21.15.96:6789/0 conn(0x7fece8135250 legacy=0x7fece8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece77fe640 1 -- 172.21.15.96:0/509405930 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece80a68b0 con 0x7fece8134e70 2024-09-13T17:44:23.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece77fe640 1 --2- 172.21.15.96:0/509405930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece809e820 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d90 tx=0x7fece001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f63ebeea49d8fe02 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece67fc640 1 -- 172.21.15.96:0/509405930 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece00a57b0 con 0x7fece8134e70 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece67fc640 1 -- 172.21.15.96:0/509405930 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00a5910 con 0x7fece8134e70 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fece67fc640 1 -- 172.21.15.96:0/509405930 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece00a5bb0 con 0x7fece8134e70 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 -- 172.21.15.96:0/509405930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 msgr2=0x7fece809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 --2- 172.21.15.96:0/509405930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece809e820 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fece0009d90 tx=0x7fece001c970 comp rx=0 tx=0).stop 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 -- 172.21.15.96:0/509405930 shutdown_connections 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 --2- 172.21.15.96:0/509405930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece809e820 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 -- 172.21.15.96:0/509405930 >> 172.21.15.96:0/509405930 conn(0x7fece8098a60 msgr2=0x7fece809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 -- 172.21.15.96:0/509405930 shutdown_connections 2024-09-13T17:44:23.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.539+0000 7fecedc15640 1 -- 172.21.15.96:0/509405930 wait complete. 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 Processor -- start 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 -- start start 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece80a2ff0 con 0x7fece8134e70 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51984/0 (socket says 172.21.15.96:51984) 2024-09-13T17:44:23.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece77fe640 1 -- 172.21.15.96:0/2108705219 learned_addr learned my addr 172.21.15.96:0/2108705219 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece77fe640 1 -- 172.21.15.96:0/2108705219 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0009a70 con 0x7fece8134e70 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece77fe640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fece00b0040 tx=0x7fece0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece00039a0 con 0x7fece8134e70 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece0003b00 con 0x7fece8134e70 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece80a0de0 con 0x7fece8134e70 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fece0003da0 con 0x7fece8134e70 2024-09-13T17:44:23.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece812faf0 con 0x7fece8134e70 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fece00b4050 con 0x7fece8134e70 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece80a0490 con 0x7fece8134e70 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 0x7fecc8037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fece00e2660 con 0x7fece8134e70 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.543+0000 7fece6ffd640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 0x7fecc8037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.547+0000 7fece6ffd640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 0x7fecc8037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fecd0001280 tx=0x7fecd0001130 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.551+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fece00ab340 con 0x7fece8134e70 2024-09-13T17:44:23.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.715+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fece81368b0 con 0x7fece8134e70 2024-09-13T17:44:23.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecdffff640 1 -- 172.21.15.96:0/2108705219 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15265 (secure 0 0 0) 0x7fece00c9b40 con 0x7fece8134e70 2024-09-13T17:44:23.723 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:23.723 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:21.568976+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 msgr2=0x7fecc8037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 0x7fecc8037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fecd0001280 tx=0x7fecd0001130 comp rx=0 tx=0).stop 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 msgr2=0x7fece80a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fece00b0040 tx=0x7fece0019710 comp rx=0 tx=0).stop 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 shutdown_connections 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fecc80352a0 0x7fecc8037760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 --2- 172.21.15.96:0/2108705219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fece8134e70 0x7fece80a5ed0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 >> 172.21.15.96:0/2108705219 conn(0x7fece8098a60 msgr2=0x7fece8131e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 shutdown_connections 2024-09-13T17:44:23.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.719+0000 7fecedc15640 1 -- 172.21.15.96:0/2108705219 wait complete. 2024-09-13T17:44:23.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:43:35.147657+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:43:35.928053+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.2200000286102295, 'score_stable': 1.2200000286102295, 'optimal_score': 1, 'raw_score_acting': 1.2200000286102295, 'raw_score_stable': 1.2200000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:23.741 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:23.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf60495640 1 Processor -- start 2024-09-13T17:44:23.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf60495640 1 -- start start 2024-09-13T17:44:23.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf60495640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcf5806ff00 0x7fcf58141340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf60495640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf580710a0 con 0x7fcf5813c4d0 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf60495640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5813c160 con 0x7fcf5806ff00 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcf5806ff00 0x7fcf58141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcf5806ff00 0x7fcf58141340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52008/0 (socket says 172.21.15.96:52008) 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 -- 172.21.15.96:0/2605412248 learned_addr learned my addr 172.21.15.96:0/2605412248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5e231640 1 --1- 172.21.15.96:0/2605412248 >> v1:172.21.15.96:6789/0 conn(0x7fcf5813c4d0 0x7fcf5813c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45852/0 (socket says 172.21.15.96:45852) 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 -- 172.21.15.96:0/2605412248 >> v1:172.21.15.96:6789/0 conn(0x7fcf5813c4d0 legacy=0x7fcf5813c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 -- 172.21.15.96:0/2605412248 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58071bd0 con 0x7fcf5806ff00 2024-09-13T17:44:23.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.859+0000 7fcf5da30640 1 --2- 172.21.15.96:0/2605412248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf58141340 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56247c93924cdc18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5d22f640 1 -- 172.21.15.96:0/2605412248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf4c0ac540 con 0x7fcf5806ff00 2024-09-13T17:44:23.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5d22f640 1 -- 172.21.15.96:0/2605412248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf4c0ac6a0 con 0x7fcf5806ff00 2024-09-13T17:44:23.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5d22f640 1 -- 172.21.15.96:0/2605412248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf4c0ac960 con 0x7fcf5806ff00 2024-09-13T17:44:23.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/2605412248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 msgr2=0x7fcf58141340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:23.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 --2- 172.21.15.96:0/2605412248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf58141340 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0097f0 tx=0x7fcf4c01c3f0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/2605412248 shutdown_connections 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 --2- 172.21.15.96:0/2605412248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf58141340 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/2605412248 >> 172.21.15.96:0/2605412248 conn(0x7fcf58098a60 msgr2=0x7fcf5809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/2605412248 shutdown_connections 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/2605412248 wait complete. 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 Processor -- start 2024-09-13T17:44:23.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- start start 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5809d420 con 0x7fcf5806ff00 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5e231640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5e231640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52018/0 (socket says 172.21.15.96:52018) 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5e231640 1 -- 172.21.15.96:0/1791237266 learned_addr learned my addr 172.21.15.96:0/1791237266 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5e231640 1 -- 172.21.15.96:0/1791237266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0094d0 con 0x7fcf5806ff00 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf5e231640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800c390 tx=0x7fcf4800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf48007410 con 0x7fcf5806ff00 2024-09-13T17:44:23.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf48007570 con 0x7fcf5806ff00 2024-09-13T17:44:23.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5813f780 con 0x7fcf5806ff00 2024-09-13T17:44:23.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf480047f0 con 0x7fcf5806ff00 2024-09-13T17:44:23.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.863+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5813bb30 con 0x7fcf5806ff00 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5809d1f0 con 0x7fcf5806ff00 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fcf48004a80 con 0x7fcf5806ff00 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf56ffd640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 0x7fcf44037710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fcf48047410 con 0x7fcf5806ff00 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf5da30640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 0x7fcf44037710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:23.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.867+0000 7fcf5da30640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 0x7fcf44037710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0ae000 tx=0x7fcf4c008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:23.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:23.871+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcf480192c0 con 0x7fcf5806ff00 2024-09-13T17:44:24.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf580a6fd0 con 0x7fcf5806ff00 2024-09-13T17:44:24.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf56ffd640 1 -- 172.21.15.96:0/1791237266 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf480113a0 con 0x7fcf5806ff00 2024-09-13T17:44:24.024 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 msgr2=0x7fcf44037710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 0x7fcf44037710 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0ae000 tx=0x7fcf4c008040 comp rx=0 tx=0).stop 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 msgr2=0x7fcf5813d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800c390 tx=0x7fcf4800c850 comp rx=0 tx=0).stop 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 shutdown_connections 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcf44035250 0x7fcf44037710 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 --2- 172.21.15.96:0/1791237266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5806ff00 0x7fcf5813d940 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 >> 172.21.15.96:0/1791237266 conn(0x7fcf58098a60 msgr2=0x7fcf581359a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.019+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 shutdown_connections 2024-09-13T17:44:24.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.023+0000 7fcf60495640 1 -- 172.21.15.96:0/1791237266 wait complete. 2024-09-13T17:44:24.041 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:44:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.155+0000 7fbbf961a640 1 Processor -- start 2024-09-13T17:44:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.155+0000 7fbbf961a640 1 -- start start 2024-09-13T17:44:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf961a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf4134e70 0x7fbbf409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf961a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf409ed60 con 0x7fbbf4135250 2024-09-13T17:44:24.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf961a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf409ddc0 con 0x7fbbf4134e70 2024-09-13T17:44:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf4134e70 0x7fbbf409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf4134e70 0x7fbbf409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52022/0 (socket says 172.21.15.96:52022) 2024-09-13T17:44:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 -- 172.21.15.96:0/2072049672 learned_addr learned my addr 172.21.15.96:0/2072049672 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 -- 172.21.15.96:0/2072049672 >> v1:172.21.15.96:6789/0 conn(0x7fbbf4135250 legacy=0x7fbbf4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 -- 172.21.15.96:0/2072049672 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf40a68b0 con 0x7fbbf4134e70 2024-09-13T17:44:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf2ffd640 1 --2- 172.21.15.96:0/2072049672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf409e820 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e6c0d90b9875b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf1ffb640 1 -- 172.21.15.96:0/2072049672 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec009200 con 0x7fbbf4134e70 2024-09-13T17:44:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf1ffb640 1 -- 172.21.15.96:0/2072049672 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec0a5750 con 0x7fbbf4134e70 2024-09-13T17:44:24.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.159+0000 7fbbf1ffb640 1 -- 172.21.15.96:0/2072049672 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0a5a10 con 0x7fbbf4134e70 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/2072049672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 msgr2=0x7fbbf409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 --2- 172.21.15.96:0/2072049672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf409e820 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec01ca30 comp rx=0 tx=0).stop 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/2072049672 shutdown_connections 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 --2- 172.21.15.96:0/2072049672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf409e820 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/2072049672 >> 172.21.15.96:0/2072049672 conn(0x7fbbf4098a60 msgr2=0x7fbbf409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/2072049672 shutdown_connections 2024-09-13T17:44:24.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/2072049672 wait complete. 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 Processor -- start 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- start start 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf40a4690 con 0x7fbbf4134e70 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52038/0 (socket says 172.21.15.96:52038) 2024-09-13T17:44:24.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf2ffd640 1 -- 172.21.15.96:0/1442661814 learned_addr learned my addr 172.21.15.96:0/1442661814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:24.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf2ffd640 1 -- 172.21.15.96:0/1442661814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec009b30 con 0x7fbbf4134e70 2024-09-13T17:44:24.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf2ffd640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbbec019640 tx=0x7fbbec0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0b8470 con 0x7fbbf4134e70 2024-09-13T17:44:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec0b85d0 con 0x7fbbf4134e70 2024-09-13T17:44:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0b8890 con 0x7fbbf4134e70 2024-09-13T17:44:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf40a5580 con 0x7fbbf4134e70 2024-09-13T17:44:24.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf40a43d0 con 0x7fbbf4134e70 2024-09-13T17:44:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fbbec0b89f0 con 0x7fbbf4134e70 2024-09-13T17:44:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf41368b0 con 0x7fbbf4134e70 2024-09-13T17:44:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 0x7fbbd4037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fbbec0e3490 con 0x7fbbf4134e70 2024-09-13T17:44:24.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.163+0000 7fbbf27fc640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 0x7fbbd4037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.167+0000 7fbbf27fc640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 0x7fbbd4037760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000f70 tx=0x7fbbdc007a40 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.171+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbbec0ac3a0 con 0x7fbbf4134e70 2024-09-13T17:44:24.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.327+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbbf4071670 con 0x7fbbf4134e70 2024-09-13T17:44:24.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.327+0000 7fbbeb7fe640 1 -- 172.21.15.96:0/1442661814 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbbec0bc020 con 0x7fbbf4134e70 2024-09-13T17:44:24.334 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 msgr2=0x7fbbd4037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 0x7fbbd4037760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc000f70 tx=0x7fbbdc007a40 comp rx=0 tx=0).stop 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 msgr2=0x7fbbf406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fbbec019640 tx=0x7fbbec0197d0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 shutdown_connections 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd40352a0 0x7fbbd4037760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 --2- 172.21.15.96:0/1442661814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf4134e70 0x7fbbf406f6d0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 >> 172.21.15.96:0/1442661814 conn(0x7fbbf4098a60 msgr2=0x7fbbf4099110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 shutdown_connections 2024-09-13T17:44:24.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.331+0000 7fbbf961a640 1 -- 172.21.15.96:0/1442661814 wait complete. 2024-09-13T17:44:24.354 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:44:24.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f4dcd8640 1 Processor -- start 2024-09-13T17:44:24.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f4dcd8640 1 -- start start 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f4dcd8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9f4809dc20 0x7f9f481351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f4dcd8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4806e130 con 0x7f9f4806f6d0 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f4dcd8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480a6750 con 0x7f9f4809dc20 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f46ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9f4809dc20 0x7f9f481351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f477fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9f4806f6d0 0x7f9f48134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45864/0 (socket says 172.21.15.96:45864) 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f477fe640 1 -- 172.21.15.96:0/566574573 learned_addr learned my addr 172.21.15.96:0/566574573 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.471+0000 7f9f46ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9f4809dc20 0x7f9f481351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52050/0 (socket says 172.21.15.96:52050) 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f46ffd640 1 -- 172.21.15.96:0/566574573 >> v1:172.21.15.96:6789/0 conn(0x7f9f4806f6d0 legacy=0x7f9f48134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f46ffd640 1 -- 172.21.15.96:0/566574573 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480712e0 con 0x7f9f4809dc20 2024-09-13T17:44:24.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f467fc640 1 -- 172.21.15.96:0/566574573 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890121473 0 0) 0x7f9f4806e130 con 0x7f9f4806f6d0 2024-09-13T17:44:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f46ffd640 1 --2- 172.21.15.96:0/566574573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4809dc20 0x7f9f481351c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800b9e0 tx=0x7f9f3801ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bd84971ba2f317 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f467fc640 1 -- 172.21.15.96:0/566574573 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f380a7750 con 0x7f9f4809dc20 2024-09-13T17:44:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f467fc640 1 -- 172.21.15.96:0/566574573 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f380a78b0 con 0x7f9f4809dc20 2024-09-13T17:44:24.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f467fc640 1 -- 172.21.15.96:0/566574573 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f380a7b70 con 0x7f9f4809dc20 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/566574573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4809dc20 msgr2=0x7f9f481351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/566574573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4809dc20 0x7f9f481351c0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800b9e0 tx=0x7f9f3801ea60 comp rx=0 tx=0).stop 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/566574573 shutdown_connections 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/566574573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4809dc20 0x7f9f481351c0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/566574573 >> 172.21.15.96:0/566574573 conn(0x7f9f48098a60 msgr2=0x7f9f4809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/566574573 shutdown_connections 2024-09-13T17:44:24.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.475+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/566574573 wait complete. 2024-09-13T17:44:24.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 Processor -- start 2024-09-13T17:44:24.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 -- start start 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f480a5d90 con 0x7f9f4806f6d0 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52054/0 (socket says 172.21.15.96:52054) 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f477fe640 1 -- 172.21.15.96:0/3151474846 learned_addr learned my addr 172.21.15.96:0/3151474846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:24.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f477fe640 1 -- 172.21.15.96:0/3151474846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3800b6c0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f477fe640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c420 tx=0x7f9f3000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f300074d0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30007630 con 0x7f9f4806f6d0 2024-09-13T17:44:24.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4809aca0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f30004960 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4809b1d0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f9f30004ac0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f48120fc0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 0x7f9f28037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f9f30046fc0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f46ffd640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 0x7f9f28037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.479+0000 7f9f46ffd640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 0x7f9f28037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9f380b1000 tx=0x7f9f380a9070 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.483+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9f3001ab20 con 0x7f9f4806f6d0 2024-09-13T17:44:24.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.575+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7f9f3001ad50 con 0x7f9f4806f6d0 2024-09-13T17:44:24.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f4806f2e0 con 0x7f9f4806f6d0 2024-09-13T17:44:24.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f3ffff640 1 -- 172.21.15.96:0/3151474846 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f30018070 con 0x7f9f4806f6d0 2024-09-13T17:44:24.656 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:24.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 msgr2=0x7f9f28037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 0x7f9f28037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9f380b1000 tx=0x7f9f380a9070 comp rx=0 tx=0).stop 2024-09-13T17:44:24.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 msgr2=0x7f9f48121250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c420 tx=0x7f9f3000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 shutdown_connections 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9f280352a0 0x7f9f28037760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 --2- 172.21.15.96:0/3151474846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4806f6d0 0x7f9f48121250 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 >> 172.21.15.96:0/3151474846 conn(0x7f9f48098a60 msgr2=0x7f9f48137af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 shutdown_connections 2024-09-13T17:44:24.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.651+0000 7f9f4dcd8640 1 -- 172.21.15.96:0/3151474846 wait complete. 2024-09-13T17:44:24.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:24.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efefa3c7640 1 Processor -- start 2024-09-13T17:44:24.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efefa3c7640 1 -- start start 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efefa3c7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efef4134e70 0x7efef409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efefa3c7640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef409ed60 con 0x7efef4135250 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efefa3c7640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef409ddc0 con 0x7efef4134e70 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efef4134e70 0x7efef409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efef4134e70 0x7efef409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52056/0 (socket says 172.21.15.96:52056) 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef3fff640 1 -- 172.21.15.96:0/3093048996 learned_addr learned my addr 172.21.15.96:0/3093048996 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:24.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef37fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efef4135250 0x7efef4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45870/0 (socket says 172.21.15.96:45870) 2024-09-13T17:44:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef3fff640 1 -- 172.21.15.96:0/3093048996 >> v1:172.21.15.96:6789/0 conn(0x7efef4135250 legacy=0x7efef4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.791+0000 7efef3fff640 1 -- 172.21.15.96:0/3093048996 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40a68b0 con 0x7efef4134e70 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 --2- 172.21.15.96:0/3093048996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409e820 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efee40097f0 tx=0x7efee401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f69d04d3a672ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef2ffd640 1 -- 172.21.15.96:0/3093048996 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efee40ac540 con 0x7efef4134e70 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef2ffd640 1 -- 172.21.15.96:0/3093048996 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee40ac6a0 con 0x7efef4134e70 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef2ffd640 1 -- 172.21.15.96:0/3093048996 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efee40ac960 con 0x7efef4134e70 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/3093048996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 msgr2=0x7efef409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 --2- 172.21.15.96:0/3093048996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409e820 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7efee40097f0 tx=0x7efee401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/3093048996 shutdown_connections 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 --2- 172.21.15.96:0/3093048996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409e820 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/3093048996 >> 172.21.15.96:0/3093048996 conn(0x7efef4098a60 msgr2=0x7efef409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/3093048996 shutdown_connections 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/3093048996 wait complete. 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 Processor -- start 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- start start 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef40a5e30 con 0x7efef4134e70 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52072/0 (socket says 172.21.15.96:52072) 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 -- 172.21.15.96:0/1977377344 learned_addr learned my addr 172.21.15.96:0/1977377344 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:24.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 -- 172.21.15.96:0/1977377344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40094d0 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef3fff640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efee40190f0 tx=0x7efee4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efee40bb070 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee40b7470 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef40a23f0 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efee40b7730 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.795+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef40a0890 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7efee40b3070 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efef0ff9640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 0x7efee0037ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef40a20e0 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7efee40e21d0 con 0x7efef4134e70 2024-09-13T17:44:24.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efef37fe640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 0x7efee0037ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:24.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.799+0000 7efef37fe640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 0x7efee0037ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efee8001600 tx=0x7efee8007a50 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:24.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.803+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efee40b1070 con 0x7efef4134e70 2024-09-13T17:44:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.959+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7efef41368b0 con 0x7efef4134e70 2024-09-13T17:44:24.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.959+0000 7efef0ff9640 1 -- 172.21.15.96:0/1977377344 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v43) v1 ==== 106+0+11 (secure 0 0 0) 0x7efee40b4380 con 0x7efef4134e70 2024-09-13T17:44:24.966 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 msgr2=0x7efee0037ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 0x7efee0037ac0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7efee8001600 tx=0x7efee8007a50 comp rx=0 tx=0).stop 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 msgr2=0x7efef409f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7efee40190f0 tx=0x7efee4019280 comp rx=0 tx=0).stop 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 shutdown_connections 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efee0035600 0x7efee0037ac0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 --2- 172.21.15.96:0/1977377344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efef4134e70 0x7efef409f280 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:24.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 >> 172.21.15.96:0/1977377344 conn(0x7efef4098a60 msgr2=0x7efef40995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:24.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 shutdown_connections 2024-09-13T17:44:24.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:24.963+0000 7efefa3c7640 1 -- 172.21.15.96:0/1977377344 wait complete. 2024-09-13T17:44:24.985 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:25.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce5b46c640 1 Processor -- start 2024-09-13T17:44:25.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce5b46c640 1 -- start start 2024-09-13T17:44:25.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce5b46c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce5409cd10 0x7fce54099170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce5b46c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5409aee0 con 0x7fce5409c930 2024-09-13T17:44:25.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce5b46c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce540a7b20 con 0x7fce5409cd10 2024-09-13T17:44:25.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce58a07640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce5409cd10 0x7fce54099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce59208640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fce5409c930 0x7fce54098a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45884/0 (socket says 172.21.15.96:45884) 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce59208640 1 -- 172.21.15.96:0/2895715485 learned_addr learned my addr 172.21.15.96:0/2895715485 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce58a07640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce5409cd10 0x7fce54099170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52076/0 (socket says 172.21.15.96:52076) 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce58a07640 1 -- 172.21.15.96:0/2895715485 >> v1:172.21.15.96:6789/0 conn(0x7fce5409c930 legacy=0x7fce54098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce4bfff640 1 -- 172.21.15.96:0/2895715485 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 186671395 0 0) 0x7fce5409aee0 con 0x7fce5409c930 2024-09-13T17:44:25.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.103+0000 7fce58a07640 1 -- 172.21.15.96:0/2895715485 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5409c090 con 0x7fce5409cd10 2024-09-13T17:44:25.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce58a07640 1 --2- 172.21.15.96:0/2895715485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409cd10 0x7fce54099170 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fce44009740 tx=0x7fce4401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5d0906ec84fbe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce4bfff640 1 -- 172.21.15.96:0/2895715485 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce440ab400 con 0x7fce5409cd10 2024-09-13T17:44:25.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce4bfff640 1 -- 172.21.15.96:0/2895715485 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440a7050 con 0x7fce5409cd10 2024-09-13T17:44:25.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce4bfff640 1 -- 172.21.15.96:0/2895715485 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce440ab6b0 con 0x7fce5409cd10 2024-09-13T17:44:25.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- 172.21.15.96:0/2895715485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409cd10 msgr2=0x7fce54099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 --2- 172.21.15.96:0/2895715485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409cd10 0x7fce54099170 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fce44009740 tx=0x7fce4401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:25.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- 172.21.15.96:0/2895715485 shutdown_connections 2024-09-13T17:44:25.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 --2- 172.21.15.96:0/2895715485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409cd10 0x7fce54099170 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- 172.21.15.96:0/2895715485 >> 172.21.15.96:0/2895715485 conn(0x7fce540668e0 msgr2=0x7fce5406ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- 172.21.15.96:0/2895715485 shutdown_connections 2024-09-13T17:44:25.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- 172.21.15.96:0/2895715485 wait complete. 2024-09-13T17:44:25.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 Processor -- start 2024-09-13T17:44:25.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.107+0000 7fce5b46c640 1 -- start start 2024-09-13T17:44:25.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce5b46c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce5b46c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce541225c0 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce59208640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce59208640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52078/0 (socket says 172.21.15.96:52078) 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce59208640 1 -- 172.21.15.96:0/3687043379 learned_addr learned my addr 172.21.15.96:0/3687043379 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce59208640 1 -- 172.21.15.96:0/3687043379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009420 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce59208640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c420 tx=0x7fce5000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce500074d0 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce50007630 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce50004960 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce54123100 con 0x7fce5409c930 2024-09-13T17:44:25.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce54120a70 con 0x7fce5409c930 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7fce50004ac0 con 0x7fce5409c930 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce54067350 con 0x7fce5409c930 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 0x7fce38037b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fce500475f0 con 0x7fce5409c930 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce58a07640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 0x7fce38037b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.111+0000 7fce58a07640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 0x7fce38037b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce440ad000 tx=0x7fce440a6000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.115+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce5001ab20 con 0x7fce5409c930 2024-09-13T17:44:25.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.271+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7fce540739a0 con 0x7fce5409c930 2024-09-13T17:44:25.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce49ffb640 1 -- 172.21.15.96:0/3687043379 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v43) v1 ==== 102+0+11 (secure 0 0 0) 0x7fce50018070 con 0x7fce5409c930 2024-09-13T17:44:25.279 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 msgr2=0x7fce38037b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 0x7fce38037b10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fce440ad000 tx=0x7fce440a6000 comp rx=0 tx=0).stop 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 msgr2=0x7fce54124800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fce5000c420 tx=0x7fce5000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 shutdown_connections 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce38035650 0x7fce38037b10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 --2- 172.21.15.96:0/3687043379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce5409c930 0x7fce54124800 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 >> 172.21.15.96:0/3687043379 conn(0x7fce540668e0 msgr2=0x7fce5406df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 shutdown_connections 2024-09-13T17:44:25.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.275+0000 7fce5b46c640 1 -- 172.21.15.96:0/3687043379 wait complete. 2024-09-13T17:44:25.298 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:25.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3049d76640 1 Processor -- start 2024-09-13T17:44:25.409 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3049d76640 1 -- start start 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3049d76640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f30441366a0 0x7f304413ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3049d76640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3044136ab0 con 0x7f30440a0860 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3049d76640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3044071cd0 con 0x7f30441366a0 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f30437fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f30440a0860 0x7f3044135f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45896/0 (socket says 172.21.15.96:45896) 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3042ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f30441366a0 0x7f304413ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f30437fe640 1 -- 172.21.15.96:0/2344900807 learned_addr learned my addr 172.21.15.96:0/2344900807 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:25.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3042ffd640 1 -- 172.21.15.96:0/2344900807 >> v1:172.21.15.96:6789/0 conn(0x7f30440a0860 legacy=0x7f3044135f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f30427fc640 1 -- 172.21.15.96:0/2344900807 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187264714 0 0) 0x7f3044136ab0 con 0x7f30440a0860 2024-09-13T17:44:25.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.403+0000 7f3042ffd640 1 -- 172.21.15.96:0/2344900807 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440a0c40 con 0x7f30441366a0 2024-09-13T17:44:25.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3042ffd640 1 --2- 172.21.15.96:0/2344900807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30441366a0 0x7f304413ae90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3038009740 tx=0x7f303801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3330b3acb8062ed1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f30427fc640 1 -- 172.21.15.96:0/2344900807 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30380ab530 con 0x7f30441366a0 2024-09-13T17:44:25.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f30427fc640 1 -- 172.21.15.96:0/2344900807 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30380a7050 con 0x7f30441366a0 2024-09-13T17:44:25.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f30427fc640 1 -- 172.21.15.96:0/2344900807 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30380ab7e0 con 0x7f30441366a0 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- 172.21.15.96:0/2344900807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30441366a0 msgr2=0x7f304413ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 --2- 172.21.15.96:0/2344900807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30441366a0 0x7f304413ae90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3038009740 tx=0x7f303801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- 172.21.15.96:0/2344900807 shutdown_connections 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 --2- 172.21.15.96:0/2344900807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30441366a0 0x7f304413ae90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- 172.21.15.96:0/2344900807 >> 172.21.15.96:0/2344900807 conn(0x7f30440a7390 msgr2=0x7f30440a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- 172.21.15.96:0/2344900807 shutdown_connections 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- 172.21.15.96:0/2344900807 wait complete. 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 Processor -- start 2024-09-13T17:44:25.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.407+0000 7f3049d76640 1 -- start start 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3049d76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3049d76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304411e3e0 con 0x7f30440a0860 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f30437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f30437fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52100/0 (socket says 172.21.15.96:52100) 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f30437fe640 1 -- 172.21.15.96:0/595750164 learned_addr learned my addr 172.21.15.96:0/595750164 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f30437fe640 1 -- 172.21.15.96:0/595750164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3038009420 con 0x7f30440a0860 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f30437fe640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f303400c420 tx=0x7f303400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30340074d0 con 0x7f30440a0860 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034007630 con 0x7f30440a0860 2024-09-13T17:44:25.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30440a62d0 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3034004960 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3044139f50 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7f3034004ac0 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3044139ce0 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 0x7f3030037af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f30340473a0 con 0x7f30440a0860 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3042ffd640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 0x7f3030037af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.411+0000 7f3042ffd640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 0x7f3030037af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30380ad000 tx=0x7f3038008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.415+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f303401c3b0 con 0x7f30440a0860 2024-09-13T17:44:25.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.603+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30440a62d0 con 0x7f30440a0860 2024-09-13T17:44:25.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.603+0000 7f302bfff640 1 -- 172.21.15.96:0/595750164 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4427 (secure 0 0 0) 0x7f30440a62d0 con 0x7f30440a0860 2024-09-13T17:44:25.610 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:25.611 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:25.611 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-09-13T17:44:25.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 msgr2=0x7f3030037af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 0x7f3030037af0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f30380ad000 tx=0x7f3038008040 comp rx=0 tx=0).stop 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 msgr2=0x7f30441cf290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f303400c420 tx=0x7f303400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 shutdown_connections 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3030035630 0x7f3030037af0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 --2- 172.21.15.96:0/595750164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30440a0860 0x7f30441cf290 secure :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f303400c420 tx=0x7f303400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 >> 172.21.15.96:0/595750164 conn(0x7f30440a7390 msgr2=0x7f3044132c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 shutdown_connections 2024-09-13T17:44:25.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.607+0000 7f3049d76640 1 -- 172.21.15.96:0/595750164 wait complete. 2024-09-13T17:44:25.628 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.739+0000 7fab3a5b6640 1 Processor -- start 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.739+0000 7fab3a5b6640 1 -- start start 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fab3409dc20 0x7fab341351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3406e130 con 0x7fab3406f6d0 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab340a6750 con 0x7fab3409dc20 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab337fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fab3409dc20 0x7fab341351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab33fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fab3406f6d0 0x7fab34134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45904/0 (socket says 172.21.15.96:45904) 2024-09-13T17:44:25.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab33fff640 1 -- 172.21.15.96:0/3123352300 learned_addr learned my addr 172.21.15.96:0/3123352300 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:25.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab337fe640 1 -- 172.21.15.96:0/3123352300 >> v1:172.21.15.96:6789/0 conn(0x7fab3406f6d0 legacy=0x7fab34134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab32ffd640 1 -- 172.21.15.96:0/3123352300 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979044746 0 0) 0x7fab3406e130 con 0x7fab3406f6d0 2024-09-13T17:44:25.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab337fe640 1 -- 172.21.15.96:0/3123352300 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab340712e0 con 0x7fab3409dc20 2024-09-13T17:44:25.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab337fe640 1 --2- 172.21.15.96:0/3123352300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3409dc20 0x7fab341351c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fab28009740 tx=0x7fab2801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77467131064a8dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab32ffd640 1 -- 172.21.15.96:0/3123352300 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab280ab400 con 0x7fab3409dc20 2024-09-13T17:44:25.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab32ffd640 1 -- 172.21.15.96:0/3123352300 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab280a7050 con 0x7fab3409dc20 2024-09-13T17:44:25.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab32ffd640 1 -- 172.21.15.96:0/3123352300 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab280ab6b0 con 0x7fab3409dc20 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- 172.21.15.96:0/3123352300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3409dc20 msgr2=0x7fab341351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/3123352300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3409dc20 0x7fab341351c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fab28009740 tx=0x7fab2801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- 172.21.15.96:0/3123352300 shutdown_connections 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/3123352300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3409dc20 0x7fab341351c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- 172.21.15.96:0/3123352300 >> 172.21.15.96:0/3123352300 conn(0x7fab34098a60 msgr2=0x7fab3409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- 172.21.15.96:0/3123352300 shutdown_connections 2024-09-13T17:44:25.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.743+0000 7fab3a5b6640 1 -- 172.21.15.96:0/3123352300 wait complete. 2024-09-13T17:44:25.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 Processor -- start 2024-09-13T17:44:25.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 -- start start 2024-09-13T17:44:25.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab341c8aa0 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52108/0 (socket says 172.21.15.96:52108) 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab33fff640 1 -- 172.21.15.96:0/841005222 learned_addr learned my addr 172.21.15.96:0/841005222 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab33fff640 1 -- 172.21.15.96:0/841005222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009420 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab33fff640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c420 tx=0x7fab2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab240074d0 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24007630 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab340a02b0 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab24004960 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab340a1b70 con 0x7fab3406f6d0 2024-09-13T17:44:25.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3409ff60 con 0x7fab3406f6d0 2024-09-13T17:44:25.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7fab24004ac0 con 0x7fab3406f6d0 2024-09-13T17:44:25.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 0x7fab20037ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:25.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fab24047530 con 0x7fab3406f6d0 2024-09-13T17:44:25.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab337fe640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 0x7fab20037ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:25.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.747+0000 7fab337fe640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 0x7fab20037ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab280ad000 tx=0x7fab28008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:25.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.751+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fab2401ab20 con 0x7fab3406f6d0 2024-09-13T17:44:25.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.911+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab34134ab0 con 0x7fab3406f6d0 2024-09-13T17:44:25.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.915+0000 7fab30ff9640 1 -- 172.21.15.96:0/841005222 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15265 (secure 0 0 0) 0x7fab24018020 con 0x7fab3406f6d0 2024-09-13T17:44:25.921 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:25.921 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:21.568976+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 msgr2=0x7fab20037ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 0x7fab20037ac0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab280ad000 tx=0x7fab28008040 comp rx=0 tx=0).stop 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 msgr2=0x7fab340a0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c420 tx=0x7fab2400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 shutdown_connections 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fab20035600 0x7fab20037ac0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 --2- 172.21.15.96:0/841005222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab3406f6d0 0x7fab340a0db0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:25.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 >> 172.21.15.96:0/841005222 conn(0x7fab34098a60 msgr2=0x7fab340702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:25.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 shutdown_connections 2024-09-13T17:44:25.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:25.919+0000 7fab3a5b6640 1 -- 172.21.15.96:0/841005222 wait complete. 2024-09-13T17:44:25.940 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:44:25.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.051+0000 7fe1f040e640 1 Processor -- start 2024-09-13T17:44:26.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.051+0000 7fe1f040e640 1 -- start start 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe1e8137fe0 0x7fe1e81383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e806b020 con 0x7fe1e8138900 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e80a0430 con 0x7fe1e8137fe0 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ee1aa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe1e8137fe0 0x7fe1e81383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed9a9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe1e8138900 0x7fe1e8134d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45914/0 (socket says 172.21.15.96:45914) 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed9a9640 1 -- 172.21.15.96:0/491169538 learned_addr learned my addr 172.21.15.96:0/491169538 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ee1aa640 1 -- 172.21.15.96:0/491169538 >> v1:172.21.15.96:6789/0 conn(0x7fe1e8138900 legacy=0x7fe1e8134d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed1a8640 1 -- 172.21.15.96:0/491169538 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156407039 0 0) 0x7fe1e806b020 con 0x7fe1e8138900 2024-09-13T17:44:26.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ee1aa640 1 -- 172.21.15.96:0/491169538 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e806ccc0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ee1aa640 1 --2- 172.21.15.96:0/491169538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e81383c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009740 tx=0x7fe1e401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28d9ae6b336122e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed1a8640 1 -- 172.21.15.96:0/491169538 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1e40a4ea0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed1a8640 1 -- 172.21.15.96:0/491169538 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e40a7050 con 0x7fe1e8137fe0 2024-09-13T17:44:26.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1ed1a8640 1 -- 172.21.15.96:0/491169538 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1e40ab680 con 0x7fe1e8137fe0 2024-09-13T17:44:26.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- 172.21.15.96:0/491169538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 msgr2=0x7fe1e81383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 --2- 172.21.15.96:0/491169538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e81383c0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009740 tx=0x7fe1e401c330 comp rx=0 tx=0).stop 2024-09-13T17:44:26.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- 172.21.15.96:0/491169538 shutdown_connections 2024-09-13T17:44:26.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 --2- 172.21.15.96:0/491169538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e81383c0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- 172.21.15.96:0/491169538 >> 172.21.15.96:0/491169538 conn(0x7fe1e806f270 msgr2=0x7fe1e8071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- 172.21.15.96:0/491169538 shutdown_connections 2024-09-13T17:44:26.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.055+0000 7fe1f040e640 1 -- 172.21.15.96:0/491169538 wait complete. 2024-09-13T17:44:26.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 Processor -- start 2024-09-13T17:44:26.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 -- start start 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e80a7220 con 0x7fe1e8137fe0 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ee1aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ee1aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52136/0 (socket says 172.21.15.96:52136) 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ee1aa640 1 -- 172.21.15.96:0/981018635 learned_addr learned my addr 172.21.15.96:0/981018635 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ee1aa640 1 -- 172.21.15.96:0/981018635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4009420 con 0x7fe1e8137fe0 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ee1aa640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4019fc0 tx=0x7fe1e401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1e40abd70 con 0x7fe1e8137fe0 2024-09-13T17:44:26.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e4004030 con 0x7fe1e8137fe0 2024-09-13T17:44:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e80a8130 con 0x7fe1e8137fe0 2024-09-13T17:44:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe1e40041e0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e80a6ea0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) v1 ==== 43455+0+0 (secure 0 0 0) 0x7fe1e40b2050 con 0x7fe1e8137fe0 2024-09-13T17:44:26.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e80a10d0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 0x7fe1cc037ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fe1e40e2290 con 0x7fe1e8137fe0 2024-09-13T17:44:26.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ed9a9640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 0x7fe1cc037ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.059+0000 7fe1ed9a9640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 0x7fe1cc037ac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.063+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe1e4004340 con 0x7fe1e8137fe0 2024-09-13T17:44:26.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.103+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fe1e4003800 con 0x7fe1e8137fe0 2024-09-13T17:44:26.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.247+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1e806a2d0 con 0x7fe1e8137fe0 2024-09-13T17:44:26.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.247+0000 7fe1deffd640 1 -- 172.21.15.96:0/981018635 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4427 (secure 0 0 0) 0x7fe1e40b0020 con 0x7fe1e8137fe0 2024-09-13T17:44:26.255 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:26.255 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:26.256 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 msgr2=0x7fe1cc037ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 0x7fe1cc037ac0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8007920 comp rx=0 tx=0).stop 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 msgr2=0x7fe1e80a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4019fc0 tx=0x7fe1e401abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 shutdown_connections 2024-09-13T17:44:26.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fe1cc035600 0x7fe1cc037ac0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 --2- 172.21.15.96:0/981018635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe1e8137fe0 0x7fe1e80a7bf0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 >> 172.21.15.96:0/981018635 conn(0x7fe1e806f270 msgr2=0x7fe1e8135d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 shutdown_connections 2024-09-13T17:44:26.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.251+0000 7fe1f040e640 1 -- 172.21.15.96:0/981018635 wait complete. 2024-09-13T17:44:26.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:26.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.383+0000 7f408b023640 1 Processor -- start 2024-09-13T17:44:26.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.383+0000 7f408b023640 1 -- start start 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4084134e70 0x7f408409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408409ed60 con 0x7f4084135250 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f408409ddc0 con 0x7f4084134e70 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4088dbf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4084134e70 0x7f408409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4083fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4084135250 0x7f4084135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45926/0 (socket says 172.21.15.96:45926) 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4083fff640 1 -- 172.21.15.96:0/2864436772 learned_addr learned my addr 172.21.15.96:0/2864436772 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4088dbf640 1 -- 172.21.15.96:0/2864436772 >> v1:172.21.15.96:6789/0 conn(0x7f4084135250 legacy=0x7f4084135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4088dbf640 1 -- 172.21.15.96:0/2864436772 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840a68b0 con 0x7f4084134e70 2024-09-13T17:44:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f4088dbf640 1 --2- 172.21.15.96:0/2864436772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408409e820 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4078009740 tx=0x7f407801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548797a951cef0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f40837fe640 1 -- 172.21.15.96:0/2864436772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40780ab530 con 0x7f4084134e70 2024-09-13T17:44:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f40837fe640 1 -- 172.21.15.96:0/2864436772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40780a7050 con 0x7f4084134e70 2024-09-13T17:44:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f40837fe640 1 -- 172.21.15.96:0/2864436772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40780ab7e0 con 0x7f4084134e70 2024-09-13T17:44:26.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- 172.21.15.96:0/2864436772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 msgr2=0x7f408409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 --2- 172.21.15.96:0/2864436772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408409e820 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f4078009740 tx=0x7f407801c420 comp rx=0 tx=0).stop 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- 172.21.15.96:0/2864436772 shutdown_connections 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 --2- 172.21.15.96:0/2864436772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408409e820 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- 172.21.15.96:0/2864436772 >> 172.21.15.96:0/2864436772 conn(0x7f4084098a60 msgr2=0x7f408409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- 172.21.15.96:0/2864436772 shutdown_connections 2024-09-13T17:44:26.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.387+0000 7f408b023640 1 -- 172.21.15.96:0/2864436772 wait complete. 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 Processor -- start 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 -- start start 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40840a1710 con 0x7f4084134e70 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4088dbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4088dbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52146/0 (socket says 172.21.15.96:52146) 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4088dbf640 1 -- 172.21.15.96:0/4079858208 learned_addr learned my addr 172.21.15.96:0/4079858208 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4088dbf640 1 -- 172.21.15.96:0/4079858208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078009420 con 0x7f4084134e70 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4088dbf640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f407801a050 tx=0x7f40780191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40780abea0 con 0x7f4084134e70 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40780b4410 con 0x7f4084134e70 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40840a3550 con 0x7f4084134e70 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40780b46b0 con 0x7f4084134e70 2024-09-13T17:44:26.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40840a4270 con 0x7f4084134e70 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40840a3350 con 0x7f4084134e70 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f40780b2040 con 0x7f4084134e70 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 0x7f4070037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f40780e2290 con 0x7f4084134e70 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4083fff640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 0x7f4070037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.391+0000 7f4083fff640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 0x7f4070037b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4074001610 tx=0x7f4074007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.395+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f40780bc3b0 con 0x7f4084134e70 2024-09-13T17:44:26.491 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:44:26.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40841368b0 con 0x7f4084134e70 2024-09-13T17:44:26.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f40817fa640 1 -- 172.21.15.96:0/4079858208 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4427 (secure 0 0 0) 0x7f40780b0020 con 0x7f4084134e70 2024-09-13T17:44:26.596 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:26.597 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:36.940692+0000","modified":"2024-09-13T17:43:38.946029+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4743},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4743":{"gid":4743,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/2227270685","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2227270685},{"type":"v1","addr":"172.21.15.96:6829","nonce":2227270685}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:26.597 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-09-13T17:44:26.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 msgr2=0x7f4070037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 0x7f4070037b50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4074001610 tx=0x7f4074007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 msgr2=0x7f408412fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f407801a050 tx=0x7f40780191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 shutdown_connections 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4070035690 0x7f4070037b50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 --2- 172.21.15.96:0/4079858208 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084134e70 0x7f408412fa60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.591+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 >> 172.21.15.96:0/4079858208 conn(0x7f4084098a60 msgr2=0x7f4084133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.595+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 shutdown_connections 2024-09-13T17:44:26.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.595+0000 7f408b023640 1 -- 172.21.15.96:0/4079858208 wait complete. 2024-09-13T17:44:26.615 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:26.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2f56e8640 1 Processor -- start 2024-09-13T17:44:26.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2f56e8640 1 -- start start 2024-09-13T17:44:26.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2f56e8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa2f009dc20 0x7fa2f01351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2f56e8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f006e130 con 0x7fa2f006f6d0 2024-09-13T17:44:26.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2f56e8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f00a6750 con 0x7fa2f009dc20 2024-09-13T17:44:26.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2ee7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa2f009dc20 0x7fa2f01351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2eeffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa2f006f6d0 0x7fa2f0134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45934/0 (socket says 172.21.15.96:45934) 2024-09-13T17:44:26.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2ee7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa2f009dc20 0x7fa2f01351c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52152/0 (socket says 172.21.15.96:52152) 2024-09-13T17:44:26.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2eeffd640 1 -- 172.21.15.96:0/4149370036 learned_addr learned my addr 172.21.15.96:0/4149370036 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:26.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2ee7fc640 1 -- 172.21.15.96:0/4149370036 >> v1:172.21.15.96:6789/0 conn(0x7fa2f006f6d0 legacy=0x7fa2f0134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.779+0000 7fa2ee7fc640 1 -- 172.21.15.96:0/4149370036 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f00712e0 con 0x7fa2f009dc20 2024-09-13T17:44:26.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2ee7fc640 1 --2- 172.21.15.96:0/4149370036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f009dc20 0x7fa2f01351c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009740 tx=0x7fa2e401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bca6bf53de41db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2edffb640 1 -- 172.21.15.96:0/4149370036 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e40ab400 con 0x7fa2f009dc20 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2edffb640 1 -- 172.21.15.96:0/4149370036 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e40a7050 con 0x7fa2f009dc20 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2edffb640 1 -- 172.21.15.96:0/4149370036 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e40ab6b0 con 0x7fa2f009dc20 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- 172.21.15.96:0/4149370036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f009dc20 msgr2=0x7fa2f01351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/4149370036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f009dc20 0x7fa2f01351c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009740 tx=0x7fa2e401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- 172.21.15.96:0/4149370036 shutdown_connections 2024-09-13T17:44:26.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/4149370036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f009dc20 0x7fa2f01351c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- 172.21.15.96:0/4149370036 >> 172.21.15.96:0/4149370036 conn(0x7fa2f0098a60 msgr2=0x7fa2f009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- 172.21.15.96:0/4149370036 shutdown_connections 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- 172.21.15.96:0/4149370036 wait complete. 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 Processor -- start 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- start start 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2f56e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00a5750 con 0x7fa2f006f6d0 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2eeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52166/0 (socket says 172.21.15.96:52166) 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2eeffd640 1 -- 172.21.15.96:0/1340280606 learned_addr learned my addr 172.21.15.96:0/1340280606 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2eeffd640 1 -- 172.21.15.96:0/1340280606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4009420 con 0x7fa2f006f6d0 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2eeffd640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800cb20 tx=0x7fa2e800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.796 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.783+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e800abf0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e800ad50 con 0x7fa2f006f6d0 2024-09-13T17:44:26.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f0139590 con 0x7fa2f006f6d0 2024-09-13T17:44:26.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2e8012400 con 0x7fa2f006f6d0 2024-09-13T17:44:26.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f00a53b0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f00a37e0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fa2e8007070 con 0x7fa2f006f6d0 2024-09-13T17:44:26.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2d77fe640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 0x7fa2dc037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:26.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7fa2e8047fe0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2ee7fc640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 0x7fa2dc037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:26.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.787+0000 7fa2ee7fc640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 0x7fa2dc037b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ad000 tx=0x7fa2e4008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:26.799 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.791+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa2e8079db0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.935+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2f0134ab0 con 0x7fa2f006f6d0 2024-09-13T17:44:26.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.939+0000 7fa2d77fe640 1 -- 172.21.15.96:0/1340280606 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+15265 (secure 0 0 0) 0x7fa2e8017070 con 0x7fa2f006f6d0 2024-09-13T17:44:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:26.945 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:21.568976+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:43:35.147657+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-13T17:43:35.928053+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 msgr2=0x7fa2dc037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 0x7fa2dc037b50 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40ad000 tx=0x7fa2e4008040 comp rx=0 tx=0).stop 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 msgr2=0x7fa2f00a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800cb20 tx=0x7fa2e800a060 comp rx=0 tx=0).stop 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 shutdown_connections 2024-09-13T17:44:26.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fa2dc035690 0x7fa2dc037b50 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 --2- 172.21.15.96:0/1340280606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa2f006f6d0 0x7fa2f00a18e0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:26.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 >> 172.21.15.96:0/1340280606 conn(0x7fa2f0098a60 msgr2=0x7fa2f01397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:26.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 shutdown_connections 2024-09-13T17:44:26.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:26.943+0000 7fa2f56e8640 1 -- 172.21.15.96:0/1340280606 wait complete. 2024-09-13T17:44:26.962 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:44:27.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.063+0000 7f6119ddc640 1 Processor -- start 2024-09-13T17:44:27.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- start start 2024-09-13T17:44:27.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6114134e70 0x7f611409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f611409ed60 con 0x7f6114135250 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f611409ddc0 con 0x7f6114134e70 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6114134e70 0x7f611409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6114134e70 0x7f611409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52168/0 (socket says 172.21.15.96:52168) 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 -- 172.21.15.96:0/1326766444 learned_addr learned my addr 172.21.15.96:0/1326766444 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:27.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6112ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6114135250 0x7f6114135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45942/0 (socket says 172.21.15.96:45942) 2024-09-13T17:44:27.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 -- 172.21.15.96:0/1326766444 >> v1:172.21.15.96:6789/0 conn(0x7f6114135250 legacy=0x7f6114135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 -- 172.21.15.96:0/1326766444 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61140a68b0 con 0x7f6114134e70 2024-09-13T17:44:27.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61137fe640 1 --2- 172.21.15.96:0/1326766444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f611409e820 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f61080097f0 tx=0x7f610801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be97a8aeb32b2047 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61127fc640 1 -- 172.21.15.96:0/1326766444 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61080ac540 con 0x7f6114134e70 2024-09-13T17:44:27.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61127fc640 1 -- 172.21.15.96:0/1326766444 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080ac6a0 con 0x7f6114134e70 2024-09-13T17:44:27.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f61127fc640 1 -- 172.21.15.96:0/1326766444 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61080ac960 con 0x7f6114134e70 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- 172.21.15.96:0/1326766444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 msgr2=0x7f611409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1326766444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f611409e820 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f61080097f0 tx=0x7f610801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- 172.21.15.96:0/1326766444 shutdown_connections 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1326766444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f611409e820 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.067+0000 7f6119ddc640 1 -- 172.21.15.96:0/1326766444 >> 172.21.15.96:0/1326766444 conn(0x7f6114098a60 msgr2=0x7f611409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- 172.21.15.96:0/1326766444 shutdown_connections 2024-09-13T17:44:27.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- 172.21.15.96:0/1326766444 wait complete. 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 Processor -- start 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- start start 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6114137f20 con 0x7f6114134e70 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f61137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f61137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52180/0 (socket says 172.21.15.96:52180) 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f61137fe640 1 -- 172.21.15.96:0/1852304668 learned_addr learned my addr 172.21.15.96:0/1852304668 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f61137fe640 1 -- 172.21.15.96:0/1852304668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61080094d0 con 0x7f6114134e70 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f61137fe640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f61080190f0 tx=0x7f6108019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61080bb070 con 0x7f6114134e70 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61080b7470 con 0x7f6114134e70 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6114132dc0 con 0x7f6114134e70 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f61080b7730 con 0x7f6114134e70 2024-09-13T17:44:27.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f611413baa0 con 0x7f6114134e70 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61141391a0 con 0x7f6114134e70 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f61080b3070 con 0x7f6114134e70 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 0x7f6100037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4987+0+0 (secure 0 0 0) 0x7f61080e22f0 con 0x7f6114134e70 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6112ffd640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 0x7f6100037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.071+0000 7f6112ffd640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 0x7f6100037b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6104001600 tx=0x7f6104007a50 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.075+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f61080b4380 con 0x7f6114134e70 2024-09-13T17:44:27.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.267+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f61141368b0 con 0x7f6114134e70 2024-09-13T17:44:27.275 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:44:27.271+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:44:27.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.279+0000 7f60fbfff640 1 -- 172.21.15.96:0/1852304668 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f61080b1090 con 0x7f6114134e70 2024-09-13T17:44:27.285 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:44:27.285 INFO:tasks.ceph.mds.b.smithi096.stderr: -4587> 2024-09-13T17:43:42.877+0000 7f52d91f0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 msgr2=0x7f6100037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 0x7f6100037b50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6104001600 tx=0x7f6104007a50 comp rx=0 tx=0).stop 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 msgr2=0x7f6114139ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f61080190f0 tx=0x7f6108019280 comp rx=0 tx=0).stop 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 shutdown_connections 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6100035690 0x7f6100037b50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 --2- 172.21.15.96:0/1852304668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6114134e70 0x7f6114139ef0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 >> 172.21.15.96:0/1852304668 conn(0x7f6114098a60 msgr2=0x7f6114098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 shutdown_connections 2024-09-13T17:44:27.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.283+0000 7f6119ddc640 1 -- 172.21.15.96:0/1852304668 wait complete. 2024-09-13T17:44:27.305 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:44:27.313 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:44:27.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 Processor -- start 2024-09-13T17:44:27.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 -- start start 2024-09-13T17:44:27.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3df81357b0 0x7f3df8139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df8134af0 con 0x7f3df809b0a0 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df809b480 con 0x7f3df81357b0 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd1eb640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3df81357b0 0x7f3df8139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd9ec640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3df809b0a0 0x7f3df81350a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45954/0 (socket says 172.21.15.96:45954) 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd9ec640 1 -- 172.21.15.96:0/3622255641 learned_addr learned my addr 172.21.15.96:0/3622255641 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd1eb640 1 -- 172.21.15.96:0/3622255641 >> v1:172.21.15.96:6789/0 conn(0x7f3df809b0a0 legacy=0x7f3df81350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd1eb640 1 -- 172.21.15.96:0/3622255641 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df809ccc0 con 0x7f3df81357b0 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfd1eb640 1 --2- 172.21.15.96:0/3622255641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df81357b0 0x7f3df8139e70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009740 tx=0x7f3de801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776b6085f7703b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfc9ea640 1 -- 172.21.15.96:0/3622255641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de80ab530 con 0x7f3df81357b0 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfc9ea640 1 -- 172.21.15.96:0/3622255641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de80a7050 con 0x7f3df81357b0 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dfc9ea640 1 -- 172.21.15.96:0/3622255641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de80ab7e0 con 0x7f3df81357b0 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 -- 172.21.15.96:0/3622255641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df81357b0 msgr2=0x7f3df8139e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.415+0000 7f3dffc50640 1 --2- 172.21.15.96:0/3622255641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df81357b0 0x7f3df8139e70 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009740 tx=0x7f3de801c330 comp rx=0 tx=0).stop 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/3622255641 shutdown_connections 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 --2- 172.21.15.96:0/3622255641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df81357b0 0x7f3df8139e70 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/3622255641 >> 172.21.15.96:0/3622255641 conn(0x7f3df806e020 msgr2=0x7f3df8070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/3622255641 shutdown_connections 2024-09-13T17:44:27.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/3622255641 wait complete. 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 Processor -- start 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- start start 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df80a3640 con 0x7f3df809b0a0 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd9ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd9ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52208/0 (socket says 172.21.15.96:52208) 2024-09-13T17:44:27.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd9ec640 1 -- 172.21.15.96:0/1986738600 learned_addr learned my addr 172.21.15.96:0/1986738600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd9ec640 1 -- 172.21.15.96:0/1986738600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de8009420 con 0x7f3df809b0a0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd9ec640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c380 tx=0x7f3df400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df4005c20 con 0x7f3df809b0a0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4005d80 con 0x7f3df809b0a0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3df4007200 con 0x7f3df809b0a0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df812d1a0 con 0x7f3df809b0a0 2024-09-13T17:44:27.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df812cce0 con 0x7f3df809b0a0 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df80a2870 con 0x7f3df809b0a0 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f3df4007360 con 0x7f3df809b0a0 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 0x7f3ddc037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5073+0+0 (secure 0 0 0) 0x7f3df4047650 con 0x7f3df809b0a0 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.419+0000 7f3dfd1eb640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 0x7f3ddc037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:27.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.423+0000 7f3dfd1eb640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 0x7f3ddc037b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3de80ad000 tx=0x7f3de8008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:27.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.427+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3df4012400 con 0x7f3df809b0a0 2024-09-13T17:44:27.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:27.603+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3df81350a0 con 0x7f3df809b0a0 2024-09-13T17:44:28.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.279+0000 7f3dee7fc640 1 -- 172.21.15.96:0/1986738600 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3df40170a0 con 0x7f3df809b0a0 2024-09-13T17:44:28.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 msgr2=0x7f3ddc037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 0x7f3ddc037b50 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3de80ad000 tx=0x7f3de8008040 comp rx=0 tx=0).stop 2024-09-13T17:44:28.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 msgr2=0x7f3df8137680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f3df400c380 tx=0x7f3df400c840 comp rx=0 tx=0).stop 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 shutdown_connections 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3ddc035690 0x7f3ddc037b50 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 --2- 172.21.15.96:0/1986738600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df809b0a0 0x7f3df8137680 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 >> 172.21.15.96:0/1986738600 conn(0x7f3df806e020 msgr2=0x7f3df8070c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 shutdown_connections 2024-09-13T17:44:28.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.283+0000 7f3dffc50640 1 -- 172.21.15.96:0/1986738600 wait complete. 2024-09-13T17:44:28.312 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:44:28.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19ed304640 1 Processor -- start 2024-09-13T17:44:28.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19ed304640 1 -- start start 2024-09-13T17:44:28.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19ed304640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19e8135e80 0x7f19e809e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19ed304640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e809e0d0 con 0x7f19e8136260 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19ed304640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e809ef50 con 0x7f19e8135e80 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19e8135e80 0x7f19e809e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e67fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f19e8136260 0x7f19e806dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45968/0 (socket says 172.21.15.96:45968) 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e67fc640 1 -- 172.21.15.96:0/52189855 learned_addr learned my addr 172.21.15.96:0/52189855 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e6ffd640 1 -- 172.21.15.96:0/52189855 >> v1:172.21.15.96:6789/0 conn(0x7f19e8136260 legacy=0x7f19e806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e6ffd640 1 -- 172.21.15.96:0/52189855 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e809bf40 con 0x7f19e8135e80 2024-09-13T17:44:28.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e5ffb640 1 -- 172.21.15.96:0/52189855 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038117056 0 0) 0x7f19e809e0d0 con 0x7f19e8136260 2024-09-13T17:44:28.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.479+0000 7f19e6ffd640 1 --2- 172.21.15.96:0/52189855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e809e890 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009740 tx=0x7f19dc01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fac5f385c926fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e5ffb640 1 -- 172.21.15.96:0/52189855 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0ab530 con 0x7f19e8135e80 2024-09-13T17:44:28.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e5ffb640 1 -- 172.21.15.96:0/52189855 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0a7050 con 0x7f19e8135e80 2024-09-13T17:44:28.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e5ffb640 1 -- 172.21.15.96:0/52189855 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0ab7e0 con 0x7f19e8135e80 2024-09-13T17:44:28.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/52189855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 msgr2=0x7f19e809e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 --2- 172.21.15.96:0/52189855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e809e890 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f19dc009740 tx=0x7f19dc01c420 comp rx=0 tx=0).stop 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/52189855 shutdown_connections 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 --2- 172.21.15.96:0/52189855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e809e890 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/52189855 >> 172.21.15.96:0/52189855 conn(0x7f19e8098a60 msgr2=0x7f19e809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/52189855 shutdown_connections 2024-09-13T17:44:28.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/52189855 wait complete. 2024-09-13T17:44:28.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 Processor -- start 2024-09-13T17:44:28.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- start start 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e81347c0 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52228/0 (socket says 172.21.15.96:52228) 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e6ffd640 1 -- 172.21.15.96:0/3632203210 learned_addr learned my addr 172.21.15.96:0/3632203210 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e6ffd640 1 -- 172.21.15.96:0/3632203210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc009420 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19e6ffd640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0ae040 tx=0x7f19dc0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0b4540 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0b46a0 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19dc0b4940 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e81349f0 con 0x7f19e8135e80 2024-09-13T17:44:28.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e809ac80 con 0x7f19e8135e80 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f19dc0b2040 con 0x7f19e8135e80 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.483+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e811f5f0 con 0x7f19e8135e80 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.487+0000 7f19cf7fe640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 0x7f19d4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.487+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5073+0+0 (secure 0 0 0) 0x7f19dc0e26b0 con 0x7f19e8135e80 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.487+0000 7f19e67fc640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 0x7f19d4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:28.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.487+0000 7f19e67fc640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 0x7f19d4037b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19d8007fd0 tx=0x7f19d8007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:28.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.491+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19dc0b3750 con 0x7f19e8135e80 2024-09-13T17:44:28.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:28.647+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f19e809ac80 con 0x7f19e8135e80 2024-09-13T17:44:29.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.287+0000 7f19cf7fe640 1 -- 172.21.15.96:0/3632203210 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f19e809ac80 con 0x7f19e8135e80 2024-09-13T17:44:29.293 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 msgr2=0x7f19d4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 0x7f19d4037b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f19d8007fd0 tx=0x7f19d8007920 comp rx=0 tx=0).stop 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 msgr2=0x7f19e811e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0ae040 tx=0x7f19dc0191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 shutdown_connections 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f19d4035690 0x7f19d4037b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:29.298 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 --2- 172.21.15.96:0/3632203210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19e8135e80 0x7f19e811e760 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:29.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 >> 172.21.15.96:0/3632203210 conn(0x7f19e8098a60 msgr2=0x7f19e8134ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:29.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 shutdown_connections 2024-09-13T17:44:29.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.291+0000 7f19ed304640 1 -- 172.21.15.96:0/3632203210 wait complete. 2024-09-13T17:44:29.323 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:44:29.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf9a88640 1 Processor -- start 2024-09-13T17:44:29.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf9a88640 1 -- start start 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf9a88640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fccf409fd20 0x7fccf413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf9a88640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf409e520 con 0x7fccf4138290 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf9a88640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf40aabe0 con 0x7fccf409fd20 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fccf409fd20 0x7fccf413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf37fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fccf4138290 0x7fccf4138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45980/0 (socket says 172.21.15.96:45980) 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf37fe640 1 -- 172.21.15.96:0/1323168052 learned_addr learned my addr 172.21.15.96:0/1323168052 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf2ffd640 1 -- 172.21.15.96:0/1323168052 >> v1:172.21.15.96:6789/0 conn(0x7fccf4138290 legacy=0x7fccf4138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf27fc640 1 -- 172.21.15.96:0/1323168052 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479253369 0 0) 0x7fccf409e520 con 0x7fccf4138290 2024-09-13T17:44:29.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.443+0000 7fccf2ffd640 1 -- 172.21.15.96:0/1323168052 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf409d610 con 0x7fccf409fd20 2024-09-13T17:44:29.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf2ffd640 1 --2- 172.21.15.96:0/1323168052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf413d1d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009740 tx=0x7fcce801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd60bfb0c012d234 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:29.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf27fc640 1 -- 172.21.15.96:0/1323168052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce80ab530 con 0x7fccf409fd20 2024-09-13T17:44:29.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf27fc640 1 -- 172.21.15.96:0/1323168052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce80a7050 con 0x7fccf409fd20 2024-09-13T17:44:29.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf27fc640 1 -- 172.21.15.96:0/1323168052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce80ab7e0 con 0x7fccf409fd20 2024-09-13T17:44:29.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- 172.21.15.96:0/1323168052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 msgr2=0x7fccf413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1323168052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf413d1d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcce8009740 tx=0x7fcce801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- 172.21.15.96:0/1323168052 shutdown_connections 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1323168052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf413d1d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- 172.21.15.96:0/1323168052 >> 172.21.15.96:0/1323168052 conn(0x7fccf406c060 msgr2=0x7fccf406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- 172.21.15.96:0/1323168052 shutdown_connections 2024-09-13T17:44:29.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- 172.21.15.96:0/1323168052 wait complete. 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 Processor -- start 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- start start 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf9a88640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf40a5f50 con 0x7fccf409fd20 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52248/0 (socket says 172.21.15.96:52248) 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf37fe640 1 -- 172.21.15.96:0/1585753121 learned_addr learned my addr 172.21.15.96:0/1585753121 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:29.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.447+0000 7fccf37fe640 1 -- 172.21.15.96:0/1585753121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce8009420 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf37fe640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c840 tx=0x7fcce400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce400a660 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce400a7e0 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf409e1a0 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcce400aac0 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf40a4010 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fcce400ac20 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf4136370 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 0x7fcce0037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4635+0+0 (secure 0 0 0) 0x7fcce40479f0 con 0x7fccf409fd20 2024-09-13T17:44:29.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf2ffd640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 0x7fcce0037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:29.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.451+0000 7fccf2ffd640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 0x7fcce0037b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcce80ad000 tx=0x7fcce80a6000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:29.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.455+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcce401a6b0 con 0x7fccf409fd20 2024-09-13T17:44:29.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:29.611+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fccf4065e90 con 0x7fccf409fd20 2024-09-13T17:44:30.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.295+0000 7fccdbfff640 1 -- 172.21.15.96:0/1585753121 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcce401a2c0 con 0x7fccf409fd20 2024-09-13T17:44:30.301 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-13T17:44:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 msgr2=0x7fcce0037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 0x7fcce0037b50 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcce80ad000 tx=0x7fcce80a6000 comp rx=0 tx=0).stop 2024-09-13T17:44:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 msgr2=0x7fccf40a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcce400c840 tx=0x7fcce400cd00 comp rx=0 tx=0).stop 2024-09-13T17:44:30.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 shutdown_connections 2024-09-13T17:44:30.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcce0035690 0x7fcce0037b50 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 --2- 172.21.15.96:0/1585753121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fccf409fd20 0x7fccf40a5a10 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 >> 172.21.15.96:0/1585753121 conn(0x7fccf406c060 msgr2=0x7fccf409a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 shutdown_connections 2024-09-13T17:44:30.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.299+0000 7fccf9a88640 1 -- 172.21.15.96:0/1585753121 wait complete. 2024-09-13T17:44:30.334 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:30.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c7d92b640 1 Processor -- start 2024-09-13T17:44:30.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c7d92b640 1 -- start start 2024-09-13T17:44:30.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c7d92b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7c7809e820 0x7f7c7813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c7d92b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c780a68b0 con 0x7f7c78134e70 2024-09-13T17:44:30.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c7d92b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7809ddc0 con 0x7f7c7809e820 2024-09-13T17:44:30.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c767fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7c7809e820 0x7f7c7813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c76ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7c78134e70 0x7f7c78135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:45994/0 (socket says 172.21.15.96:45994) 2024-09-13T17:44:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c76ffd640 1 -- 172.21.15.96:0/1480133912 learned_addr learned my addr 172.21.15.96:0/1480133912 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c767fc640 1 -- 172.21.15.96:0/1480133912 >> v1:172.21.15.96:6789/0 conn(0x7f7c78134e70 legacy=0x7f7c78135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c767fc640 1 -- 172.21.15.96:0/1480133912 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c78136970 con 0x7f7c7809e820 2024-09-13T17:44:30.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.447+0000 7f7c75ffb640 1 -- 172.21.15.96:0/1480133912 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942530223 0 0) 0x7f7c780a68b0 con 0x7f7c78134e70 2024-09-13T17:44:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c767fc640 1 --2- 172.21.15.96:0/1480133912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7813ae50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009740 tx=0x7f7c6c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e69582bf292c409 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c75ffb640 1 -- 172.21.15.96:0/1480133912 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0ab530 con 0x7f7c7809e820 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c75ffb640 1 -- 172.21.15.96:0/1480133912 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6c0a7050 con 0x7f7c7809e820 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c75ffb640 1 -- 172.21.15.96:0/1480133912 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c6c0ab7e0 con 0x7f7c7809e820 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1480133912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 msgr2=0x7f7c7813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1480133912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7813ae50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c009740 tx=0x7f7c6c01c420 comp rx=0 tx=0).stop 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1480133912 shutdown_connections 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1480133912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7813ae50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1480133912 >> 172.21.15.96:0/1480133912 conn(0x7f7c78098a60 msgr2=0x7f7c7809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1480133912 shutdown_connections 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1480133912 wait complete. 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 Processor -- start 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- start start 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c7d92b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c780a1200 con 0x7f7c7809e820 2024-09-13T17:44:30.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.451+0000 7f7c76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52266/0 (socket says 172.21.15.96:52266) 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c76ffd640 1 -- 172.21.15.96:0/1434748992 learned_addr learned my addr 172.21.15.96:0/1434748992 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c76ffd640 1 -- 172.21.15.96:0/1434748992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c009420 con 0x7f7c7809e820 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c76ffd640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c420 tx=0x7f7c6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c680074d0 con 0x7f7c7809e820 2024-09-13T17:44:30.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68007630 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c780a6150 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c68004960 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c780a1900 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f7c68004ac0 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c780a0a80 con 0x7f7c7809e820 2024-09-13T17:44:30.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 0x7f7c64037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f7c68047380 con 0x7f7c7809e820 2024-09-13T17:44:30.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c767fc640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 0x7f7c64037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.455+0000 7f7c767fc640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 0x7f7c64037b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0ad000 tx=0x7f7c6c008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.459+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7c6801acd0 con 0x7f7c7809e820 2024-09-13T17:44:30.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.619+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c780a62e0 con 0x7f7c7809e820 2024-09-13T17:44:30.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.619+0000 7f7c5ffff640 1 -- 172.21.15.96:0/1434748992 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+11627 (secure 0 0 0) 0x7f7c68018070 con 0x7f7c7809e820 2024-09-13T17:44:30.626 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:30.626 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":46,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:30.289043+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6829/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6828/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 msgr2=0x7f7c64037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 0x7f7c64037b50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7c6c0ad000 tx=0x7f7c6c008040 comp rx=0 tx=0).stop 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 msgr2=0x7f7c7809f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c420 tx=0x7f7c6800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 shutdown_connections 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7c64035690 0x7f7c64037b50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 --2- 172.21.15.96:0/1434748992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c7809e820 0x7f7c7809f280 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 >> 172.21.15.96:0/1434748992 conn(0x7f7c78098a60 msgr2=0x7f7c7806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 shutdown_connections 2024-09-13T17:44:30.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.623+0000 7f7c7d92b640 1 -- 172.21.15.96:0/1434748992 wait complete. 2024-09-13T17:44:30.645 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_b_metadata', 'create_time': '2024-09-13T17:43:54.524016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_b_data', 'create_time': '2024-09-13T17:43:54.975952+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs_b'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:30.646 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:30.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.763+0000 7f7722e91640 1 Processor -- start 2024-09-13T17:44:30.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.763+0000 7f7722e91640 1 -- start start 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7722e91640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f771c06f6d0 0x7f771c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7722e91640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c06e130 con 0x7f771c09dc20 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7722e91640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c0a6750 con 0x7f771c06f6d0 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f771c06f6d0 0x7f771c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f771c06f6d0 0x7f771c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52282/0 (socket says 172.21.15.96:52282) 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 -- 172.21.15.96:0/793511193 learned_addr learned my addr 172.21.15.96:0/793511193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f771bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f771c09dc20 0x7f771c134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46004/0 (socket says 172.21.15.96:46004) 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 -- 172.21.15.96:0/793511193 >> v1:172.21.15.96:6789/0 conn(0x7f771c09dc20 legacy=0x7f771c134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 -- 172.21.15.96:0/793511193 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c0712e0 con 0x7f771c06f6d0 2024-09-13T17:44:30.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f771b7fe640 1 -- 172.21.15.96:0/793511193 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318920324 0 0) 0x7f771c06e130 con 0x7f771c09dc20 2024-09-13T17:44:30.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f7720c2d640 1 --2- 172.21.15.96:0/793511193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c134ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f770c009e50 tx=0x7f770c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efe435af1f860ee server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f771b7fe640 1 -- 172.21.15.96:0/793511193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f770c0090d0 con 0x7f771c06f6d0 2024-09-13T17:44:30.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f771b7fe640 1 -- 172.21.15.96:0/793511193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c009230 con 0x7f771c06f6d0 2024-09-13T17:44:30.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.767+0000 7f771b7fe640 1 -- 172.21.15.96:0/793511193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f770c0a5850 con 0x7f771c06f6d0 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/793511193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 msgr2=0x7f771c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 --2- 172.21.15.96:0/793511193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c134ab0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f770c009e50 tx=0x7f770c01c9a0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/793511193 shutdown_connections 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 --2- 172.21.15.96:0/793511193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c134ab0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/793511193 >> 172.21.15.96:0/793511193 conn(0x7f771c098a60 msgr2=0x7f771c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/793511193 shutdown_connections 2024-09-13T17:44:30.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/793511193 wait complete. 2024-09-13T17:44:30.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 Processor -- start 2024-09-13T17:44:30.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- start start 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c0a36f0 con 0x7f771c06f6d0 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7720c2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7720c2d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52298/0 (socket says 172.21.15.96:52298) 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7720c2d640 1 -- 172.21.15.96:0/779306655 learned_addr learned my addr 172.21.15.96:0/779306655 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7720c2d640 1 -- 172.21.15.96:0/779306655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c009b30 con 0x7f771c06f6d0 2024-09-13T17:44:30.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7720c2d640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f770c0195b0 tx=0x7f770c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f770c009510 con 0x7f771c06f6d0 2024-09-13T17:44:30.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c0b8680 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c0a2d70 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f770c0b8920 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c06f340 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c0a1d90 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.771+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f770c0b8a80 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.775+0000 7f77197fa640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 0x7f7708037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.775+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f770c0e3670 con 0x7f771c06f6d0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.775+0000 7f771bfff640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 0x7f7708037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:30.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.775+0000 7f771bfff640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 0x7f7708037b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7710007fd0 tx=0x7f7710007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:30.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.779+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f770c0ac3a0 con 0x7f771c06f6d0 2024-09-13T17:44:30.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f771c132e50 con 0x7f771c06f6d0 2024-09-13T17:44:30.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f77197fa640 1 -- 172.21.15.96:0/779306655 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f770c0bc020 con 0x7f771c06f6d0 2024-09-13T17:44:30.944 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 msgr2=0x7f7708037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 0x7f7708037b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7710007fd0 tx=0x7f7710007920 comp rx=0 tx=0).stop 2024-09-13T17:44:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 msgr2=0x7f771c0a1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f770c0195b0 tx=0x7f770c019740 comp rx=0 tx=0).stop 2024-09-13T17:44:30.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 shutdown_connections 2024-09-13T17:44:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7708035690 0x7f7708037b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 --2- 172.21.15.96:0/779306655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f771c06f6d0 0x7f771c0a1f90 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 >> 172.21.15.96:0/779306655 conn(0x7f771c098a60 msgr2=0x7f771c131d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 shutdown_connections 2024-09-13T17:44:30.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:30.939+0000 7f7722e91640 1 -- 172.21.15.96:0/779306655 wait complete. 2024-09-13T17:44:30.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_metadata pg_num 2024-09-13T17:44:31.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.075+0000 7f4ff8d29640 1 Processor -- start 2024-09-13T17:44:31.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff8d29640 1 -- start start 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff8d29640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4ff4134e70 0x7f4ff409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff8d29640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff409ed60 con 0x7f4ff4135250 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff8d29640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff409ddc0 con 0x7f4ff4134e70 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4ff4134e70 0x7f4ff409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4fe9d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4ff4135250 0x7f4ff4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46016/0 (socket says 172.21.15.96:46016) 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4ff4134e70 0x7f4ff409e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52304/0 (socket says 172.21.15.96:52304) 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4fe9d9b640 1 -- 172.21.15.96:0/3683348984 learned_addr learned my addr 172.21.15.96:0/3683348984 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff259c640 1 -- 172.21.15.96:0/3683348984 >> v1:172.21.15.96:6789/0 conn(0x7f4ff4135250 legacy=0x7f4ff4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff259c640 1 -- 172.21.15.96:0/3683348984 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff40a68b0 con 0x7f4ff4134e70 2024-09-13T17:44:31.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff1d9b640 1 -- 172.21.15.96:0/3683348984 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042054876 0 0) 0x7f4fdc001120 con 0x7f4ff4135250 2024-09-13T17:44:31.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff259c640 1 --2- 172.21.15.96:0/3683348984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff409e820 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009740 tx=0x7f4fe401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5372b841d3746546 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff1d9b640 1 -- 172.21.15.96:0/3683348984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40ab530 con 0x7f4ff4134e70 2024-09-13T17:44:31.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff1d9b640 1 -- 172.21.15.96:0/3683348984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe40a7050 con 0x7f4ff4134e70 2024-09-13T17:44:31.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.079+0000 7f4ff1d9b640 1 -- 172.21.15.96:0/3683348984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40ab7e0 con 0x7f4ff4134e70 2024-09-13T17:44:31.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3683348984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 msgr2=0x7f4ff409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3683348984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff409e820 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4009740 tx=0x7f4fe401c420 comp rx=0 tx=0).stop 2024-09-13T17:44:31.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3683348984 shutdown_connections 2024-09-13T17:44:31.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3683348984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff409e820 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3683348984 >> 172.21.15.96:0/3683348984 conn(0x7f4ff4098a60 msgr2=0x7f4ff409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3683348984 shutdown_connections 2024-09-13T17:44:31.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3683348984 wait complete. 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 Processor -- start 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- start start 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff413bb90 con 0x7f4ff4134e70 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52310/0 (socket says 172.21.15.96:52310) 2024-09-13T17:44:31.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff259c640 1 -- 172.21.15.96:0/3270254746 learned_addr learned my addr 172.21.15.96:0/3270254746 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:31.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff259c640 1 -- 172.21.15.96:0/3270254746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009420 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff259c640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4019030 tx=0x7f4fe401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40abde0 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe4004030 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4fe40042d0 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff40a2e20 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff40a1b90 con 0x7f4ff4134e70 2024-09-13T17:44:31.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f4fe4003620 con 0x7f4ff4134e70 2024-09-13T17:44:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.083+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff41368b0 con 0x7f4ff4134e70 2024-09-13T17:44:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.087+0000 7f4feb7fe640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 0x7f4fd4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.087+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f4fe40e2ba0 con 0x7f4ff4134e70 2024-09-13T17:44:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.087+0000 7f4fe9d9b640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 0x7f4fd4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.087+0000 7f4fe9d9b640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 0x7f4fd4037b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000c10 tx=0x7f4fdc001320 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.091+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4fe40aa970 con 0x7f4ff4134e70 2024-09-13T17:44:31.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ff40a2bf0 con 0x7f4ff4134e70 2024-09-13T17:44:31.252 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4feb7fe640 1 -- 172.21.15.96:0/3270254746 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_metadata", "var": "pg_num"}]=0 v46) v1 ==== 106+0+11 (secure 0 0 0) 0x7f4fe40b0030 con 0x7f4ff4134e70 2024-09-13T17:44:31.252 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:44:31.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 msgr2=0x7f4fd4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 0x7f4fd4037b50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc000c10 tx=0x7f4fdc001320 comp rx=0 tx=0).stop 2024-09-13T17:44:31.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 msgr2=0x7f4ff40a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f4fe4019030 tx=0x7f4fe401ac60 comp rx=0 tx=0).stop 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 shutdown_connections 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4fd4035690 0x7f4fd4037b50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 --2- 172.21.15.96:0/3270254746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ff4134e70 0x7f4ff40a25f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 >> 172.21.15.96:0/3270254746 conn(0x7f4ff4098a60 msgr2=0x7f4ff4131da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 shutdown_connections 2024-09-13T17:44:31.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.247+0000 7f4ff8d29640 1 -- 172.21.15.96:0/3270254746 wait complete. 2024-09-13T17:44:31.270 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_b_data pg_num 2024-09-13T17:44:31.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.391+0000 7f8d3c911640 1 Processor -- start 2024-09-13T17:44:31.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.391+0000 7f8d3c911640 1 -- start start 2024-09-13T17:44:31.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8d3809dc20 0x7f8d381351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3806e130 con 0x7f8d3806f6d0 2024-09-13T17:44:31.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d380a6750 con 0x7f8d3809dc20 2024-09-13T17:44:31.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d2dd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8d3809dc20 0x7f8d381351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8d3806f6d0 0x7f8d38134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46032/0 (socket says 172.21.15.96:46032) 2024-09-13T17:44:31.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3659c640 1 -- 172.21.15.96:0/3109455771 learned_addr learned my addr 172.21.15.96:0/3109455771 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:31.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d2dd9b640 1 -- 172.21.15.96:0/3109455771 >> v1:172.21.15.96:6789/0 conn(0x7f8d3806f6d0 legacy=0x7f8d38134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d2dd9b640 1 -- 172.21.15.96:0/3109455771 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380712e0 con 0x7f8d3809dc20 2024-09-13T17:44:31.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d35d9b640 1 -- 172.21.15.96:0/3109455771 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10844388 0 0) 0x7f8d3806e130 con 0x7f8d3806f6d0 2024-09-13T17:44:31.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d2dd9b640 1 --2- 172.21.15.96:0/3109455771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3809dc20 0x7f8d381351c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009740 tx=0x7f8d2001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d70d3439ed1067 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d35d9b640 1 -- 172.21.15.96:0/3109455771 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d200ab400 con 0x7f8d3809dc20 2024-09-13T17:44:31.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d35d9b640 1 -- 172.21.15.96:0/3109455771 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d200a7050 con 0x7f8d3809dc20 2024-09-13T17:44:31.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d35d9b640 1 -- 172.21.15.96:0/3109455771 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d200ab6b0 con 0x7f8d3809dc20 2024-09-13T17:44:31.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- 172.21.15.96:0/3109455771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3809dc20 msgr2=0x7f8d381351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 --2- 172.21.15.96:0/3109455771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3809dc20 0x7f8d381351c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8d20009740 tx=0x7f8d2001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- 172.21.15.96:0/3109455771 shutdown_connections 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 --2- 172.21.15.96:0/3109455771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3809dc20 0x7f8d381351c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- 172.21.15.96:0/3109455771 >> 172.21.15.96:0/3109455771 conn(0x7f8d38098a60 msgr2=0x7f8d3809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- 172.21.15.96:0/3109455771 shutdown_connections 2024-09-13T17:44:31.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.395+0000 7f8d3c911640 1 -- 172.21.15.96:0/3109455771 wait complete. 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 Processor -- start 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 -- start start 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38120b10 con 0x7f8d3806f6d0 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52328/0 (socket says 172.21.15.96:52328) 2024-09-13T17:44:31.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3659c640 1 -- 172.21.15.96:0/925231508 learned_addr learned my addr 172.21.15.96:0/925231508 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:31.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3659c640 1 -- 172.21.15.96:0/925231508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d20009420 con 0x7f8d3806f6d0 2024-09-13T17:44:31.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3659c640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c420 tx=0x7f8d2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d280074d0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d28007630 con 0x7f8d3806f6d0 2024-09-13T17:44:31.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d38121400 con 0x7f8d3806f6d0 2024-09-13T17:44:31.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d28004960 con 0x7f8d3806f6d0 2024-09-13T17:44:31.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d381204e0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f8d28004ac0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d380991f0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 0x7f8d18037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f8d28047380 con 0x7f8d3806f6d0 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.399+0000 7f8d2dd9b640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 0x7f8d18037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.403+0000 7f8d2dd9b640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 0x7f8d18037b50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d200ad000 tx=0x7f8d20008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.407+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8d2801acd0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"} v 0) v1 -- 0x7f8d38134ab0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d2f7fe640 1 -- 172.21.15.96:0/925231508 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_b_data", "var": "pg_num"}]=0 v46) v1 ==== 102+0+11 (secure 0 0 0) 0x7f8d280180a0 con 0x7f8d3806f6d0 2024-09-13T17:44:31.568 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 msgr2=0x7f8d18037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d3c911640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 0x7f8d18037b50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8d200ad000 tx=0x7f8d20008040 comp rx=0 tx=0).stop 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 msgr2=0x7f8d3811d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.563+0000 7f8d3c911640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f8d2800c420 tx=0x7f8d2800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 shutdown_connections 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8d18035690 0x7f8d18037b50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 --2- 172.21.15.96:0/925231508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d3806f6d0 0x7f8d3811d060 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 >> 172.21.15.96:0/925231508 conn(0x7f8d38098a60 msgr2=0x7f8d380a5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 shutdown_connections 2024-09-13T17:44:31.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.567+0000 7f8d3c911640 1 -- 172.21.15.96:0/925231508 wait complete. 2024-09-13T17:44:31.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:31.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f1082240640 1 Processor -- start 2024-09-13T17:44:31.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f1082240640 1 -- start start 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f1082240640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f107c134e70 0x7f107c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f1082240640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c09ed60 con 0x7f107c135250 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f1082240640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107c09ddc0 con 0x7f107c134e70 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f107b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f107c134e70 0x7f107c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f107affd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f107c135250 0x7f107c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46046/0 (socket says 172.21.15.96:46046) 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f107affd640 1 -- 172.21.15.96:0/1117232442 learned_addr learned my addr 172.21.15.96:0/1117232442 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.699+0000 7f107b7fe640 1 -- 172.21.15.96:0/1117232442 >> v1:172.21.15.96:6789/0 conn(0x7f107c135250 legacy=0x7f107c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107b7fe640 1 -- 172.21.15.96:0/1117232442 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0a68b0 con 0x7f107c134e70 2024-09-13T17:44:31.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107a7fc640 1 -- 172.21.15.96:0/1117232442 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703146205 0 0) 0x7f1064001120 con 0x7f107c135250 2024-09-13T17:44:31.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107b7fe640 1 --2- 172.21.15.96:0/1117232442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c09e820 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f106c009740 tx=0x7f106c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b77af6b6f656b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107a7fc640 1 -- 172.21.15.96:0/1117232442 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0ab530 con 0x7f107c134e70 2024-09-13T17:44:31.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107a7fc640 1 -- 172.21.15.96:0/1117232442 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0a7050 con 0x7f107c134e70 2024-09-13T17:44:31.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f107a7fc640 1 -- 172.21.15.96:0/1117232442 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0ab7e0 con 0x7f107c134e70 2024-09-13T17:44:31.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- 172.21.15.96:0/1117232442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 msgr2=0x7f107c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 --2- 172.21.15.96:0/1117232442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c09e820 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f106c009740 tx=0x7f106c01c420 comp rx=0 tx=0).stop 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- 172.21.15.96:0/1117232442 shutdown_connections 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 --2- 172.21.15.96:0/1117232442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c09e820 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- 172.21.15.96:0/1117232442 >> 172.21.15.96:0/1117232442 conn(0x7f107c098a60 msgr2=0x7f107c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- 172.21.15.96:0/1117232442 shutdown_connections 2024-09-13T17:44:31.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- 172.21.15.96:0/1117232442 wait complete. 2024-09-13T17:44:31.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 Processor -- start 2024-09-13T17:44:31.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.703+0000 7f1082240640 1 -- start start 2024-09-13T17:44:31.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1082240640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1082240640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0a4f90 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52340/0 (socket says 172.21.15.96:52340) 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107b7fe640 1 -- 172.21.15.96:0/2083217929 learned_addr learned my addr 172.21.15.96:0/2083217929 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107b7fe640 1 -- 172.21.15.96:0/2083217929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c009420 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107b7fe640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f106c019030 tx=0x7f106c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0b7450 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0b75b0 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f106c0b7890 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c070800 con 0x7f107c134e70 2024-09-13T17:44:31.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c0711e0 con 0x7f107c134e70 2024-09-13T17:44:31.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f106c0b79f0 con 0x7f107c134e70 2024-09-13T17:44:31.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c1368b0 con 0x7f107c134e70 2024-09-13T17:44:31.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 0x7f105c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:31.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f106c0e2840 con 0x7f107c134e70 2024-09-13T17:44:31.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107affd640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 0x7f105c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:31.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.707+0000 7f107affd640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 0x7f105c037b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1064000c10 tx=0x7f1064001320 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:31.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.711+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f106c0c74e0 con 0x7f107c134e70 2024-09-13T17:44:31.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.903+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f107c071530 con 0x7f107c134e70 2024-09-13T17:44:31.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.903+0000 7f1073fff640 1 -- 172.21.15.96:0/2083217929 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3220 (secure 0 0 0) 0x7f106c0bb090 con 0x7f107c134e70 2024-09-13T17:44:31.910 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:31.911 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/1651635905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1651635905},{"type":"v1","addr":"172.21.15.96:6829","nonce":1651635905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:31.911 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-13T17:44:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 msgr2=0x7f105c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 0x7f105c037b50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1064000c10 tx=0x7f1064001320 comp rx=0 tx=0).stop 2024-09-13T17:44:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 msgr2=0x7f107c06f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:31.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f106c019030 tx=0x7f106c01ac60 comp rx=0 tx=0).stop 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 shutdown_connections 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f105c035690 0x7f105c037b50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 --2- 172.21.15.96:0/2083217929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f107c134e70 0x7f107c06f6d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 >> 172.21.15.96:0/2083217929 conn(0x7f107c098a60 msgr2=0x7f107c098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 shutdown_connections 2024-09-13T17:44:31.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:31.907+0000 7f1082240640 1 -- 172.21.15.96:0/2083217929 wait complete. 2024-09-13T17:44:31.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:32.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd8681640 1 Processor -- start 2024-09-13T17:44:32.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd8681640 1 -- start start 2024-09-13T17:44:32.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd8681640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbd009e820 0x7fdbd013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd8681640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd00a68b0 con 0x7fdbd0134e70 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd8681640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd009ddc0 con 0x7fdbd009e820 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd5c1c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbd009e820 0x7fdbd013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd641d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdbd0134e70 0x7fdbd0135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46056/0 (socket says 172.21.15.96:46056) 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd641d640 1 -- 172.21.15.96:0/1913853813 learned_addr learned my addr 172.21.15.96:0/1913853813 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd5c1c640 1 -- 172.21.15.96:0/1913853813 >> v1:172.21.15.96:6789/0 conn(0x7fdbd0134e70 legacy=0x7fdbd0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd5c1c640 1 -- 172.21.15.96:0/1913853813 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0136970 con 0x7fdbd009e820 2024-09-13T17:44:32.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.039+0000 7fdbd541b640 1 -- 172.21.15.96:0/1913853813 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197921855 0 0) 0x7fdbd00a68b0 con 0x7fdbd0134e70 2024-09-13T17:44:32.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd5c1c640 1 --2- 172.21.15.96:0/1913853813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd013ae50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009740 tx=0x7fdbc001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677d48e30684ed04 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd541b640 1 -- 172.21.15.96:0/1913853813 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc00ab530 con 0x7fdbd009e820 2024-09-13T17:44:32.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd541b640 1 -- 172.21.15.96:0/1913853813 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc00a7050 con 0x7fdbd009e820 2024-09-13T17:44:32.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd541b640 1 -- 172.21.15.96:0/1913853813 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc00ab7e0 con 0x7fdbd009e820 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/1913853813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 msgr2=0x7fdbd013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 --2- 172.21.15.96:0/1913853813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd013ae50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0009740 tx=0x7fdbc001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/1913853813 shutdown_connections 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 --2- 172.21.15.96:0/1913853813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd013ae50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/1913853813 >> 172.21.15.96:0/1913853813 conn(0x7fdbd0098a60 msgr2=0x7fdbd009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/1913853813 shutdown_connections 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/1913853813 wait complete. 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 Processor -- start 2024-09-13T17:44:32.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- start start 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd011ca40 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd641d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd641d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52352/0 (socket says 172.21.15.96:52352) 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd641d640 1 -- 172.21.15.96:0/4085194020 learned_addr learned my addr 172.21.15.96:0/4085194020 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd641d640 1 -- 172.21.15.96:0/4085194020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc0009420 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd641d640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c420 tx=0x7fdbcc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc0074d0 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc007630 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd011d6e0 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc004960 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.043+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd013a8f0 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fdbcc004ac0 con 0x7fdbd009e820 2024-09-13T17:44:32.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbc6ffd640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 0x7fdbb4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00a45e0 con 0x7fdbd009e820 2024-09-13T17:44:32.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7fdbcc047340 con 0x7fdbd009e820 2024-09-13T17:44:32.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbd5c1c640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 0x7fdbb4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.047+0000 7fdbd5c1c640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 0x7fdbb4037b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ad000 tx=0x7fdbc0008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.051+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdbcc018030 con 0x7fdbd009e820 2024-09-13T17:44:32.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.199+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdbd013a430 con 0x7fdbd009e820 2024-09-13T17:44:32.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.199+0000 7fdbc6ffd640 1 -- 172.21.15.96:0/4085194020 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+11627 (secure 0 0 0) 0x7fdbcc018030 con 0x7fdbd009e820 2024-09-13T17:44:32.206 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:32.206 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":46,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:30.289043+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6829/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6828/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:32.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 msgr2=0x7fdbb4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 0x7fdbb4037b50 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdbc00ad000 tx=0x7fdbc0008040 comp rx=0 tx=0).stop 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 msgr2=0x7fdbd00a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00c420 tx=0x7fdbcc00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 shutdown_connections 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fdbb4035690 0x7fdbb4037b50 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 --2- 172.21.15.96:0/4085194020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd009e820 0x7fdbd00a26c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 >> 172.21.15.96:0/4085194020 conn(0x7fdbd0098a60 msgr2=0x7fdbd0133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 shutdown_connections 2024-09-13T17:44:32.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.203+0000 7fdbd8681640 1 -- 172.21.15.96:0/4085194020 wait complete. 2024-09-13T17:44:32.224 INFO:tasks.cephfs.filesystem:Destroying file system cephfs_b and related pools 2024-09-13T17:44:32.225 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:32.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.339+0000 7f4ea0437640 1 Processor -- start 2024-09-13T17:44:32.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.339+0000 7f4ea0437640 1 -- start start 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e98135140 0x7f4e98135520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9812ef00 con 0x7f4e9809c960 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e98099ec0 con 0x7f4e98135140 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9e1d3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e98135140 0x7f4e98135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d9d2640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4e9809c960 0x7f4e98131440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:46058/0 (socket says 172.21.15.96:46058) 2024-09-13T17:44:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9e1d3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4e98135140 0x7f4e98135520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52366/0 (socket says 172.21.15.96:52366) 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d9d2640 1 -- 172.21.15.96:0/3951280696 learned_addr learned my addr 172.21.15.96:0/3951280696 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9e1d3640 1 -- 172.21.15.96:0/3951280696 >> v1:172.21.15.96:6789/0 conn(0x7f4e9809c960 legacy=0x7f4e98131440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9e1d3640 1 -- 172.21.15.96:0/3951280696 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9809bf20 con 0x7f4e98135140 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d1d1640 1 -- 172.21.15.96:0/3951280696 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265574167 0 0) 0x7f4e88001120 con 0x7f4e9809c960 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9e1d3640 1 --2- 172.21.15.96:0/3951280696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e98135140 0x7f4e98135520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009d90 tx=0x7f4e9001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d14b73681e4f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d1d1640 1 -- 172.21.15.96:0/3951280696 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e98132360 con 0x7f4e98135140 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d1d1640 1 -- 172.21.15.96:0/3951280696 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e900a5790 con 0x7f4e98135140 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4e9d1d1640 1 -- 172.21.15.96:0/3951280696 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e900a5a50 con 0x7f4e98135140 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- 172.21.15.96:0/3951280696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e98135140 msgr2=0x7f4e98135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 --2- 172.21.15.96:0/3951280696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e98135140 0x7f4e98135520 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009d90 tx=0x7f4e9001c970 comp rx=0 tx=0).stop 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- 172.21.15.96:0/3951280696 shutdown_connections 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 --2- 172.21.15.96:0/3951280696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e98135140 0x7f4e98135520 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- 172.21.15.96:0/3951280696 >> 172.21.15.96:0/3951280696 conn(0x7f4e9806e020 msgr2=0x7f4e98070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- 172.21.15.96:0/3951280696 shutdown_connections 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.343+0000 7f4ea0437640 1 -- 172.21.15.96:0/3951280696 wait complete. 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 Processor -- start 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 -- start start 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e981cc1b0 con 0x7f4e9809c960 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9e1d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9e1d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52384/0 (socket says 172.21.15.96:52384) 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9e1d3640 1 -- 172.21.15.96:0/2303040061 learned_addr learned my addr 172.21.15.96:0/2303040061 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9e1d3640 1 -- 172.21.15.96:0/2303040061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e90009a70 con 0x7f4e9809c960 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9e1d3640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4e9001c9b0 tx=0x7f4e90019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e900ad780 con 0x7f4e9809c960 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e900ad8e0 con 0x7f4e9809c960 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e980a24d0 con 0x7f4e9809c960 2024-09-13T17:44:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4e900adb80 con 0x7f4e9809c960 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e981cc3a0 con 0x7f4e9809c960 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e980a2db0 con 0x7f4e9809c960 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f4e900b4050 con 0x7f4e9809c960 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 0x7f4e7c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f4e900e2bb0 con 0x7f4e9809c960 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.347+0000 7f4e9d9d2640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 0x7f4e7c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.351+0000 7f4e9d9d2640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 0x7f4e7c037b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c10 tx=0x7f4e88001320 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.355+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f4e900ac990 con 0x7f4e9809c960 2024-09-13T17:44:32.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.531+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e98073a50 con 0x7f4e9809c960 2024-09-13T17:44:32.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.531+0000 7f4e8effd640 1 -- 172.21.15.96:0/2303040061 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3220 (secure 0 0 0) 0x7f4e900b2030 con 0x7f4e9809c960 2024-09-13T17:44:32.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:32.538 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/1651635905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1651635905},{"type":"v1","addr":"172.21.15.96:6829","nonce":1651635905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:32.538 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-13T17:44:32.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 msgr2=0x7f4e7c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 0x7f4e7c037b50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4e88000c10 tx=0x7f4e88001320 comp rx=0 tx=0).stop 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 msgr2=0x7f4e981cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4e9001c9b0 tx=0x7f4e90019710 comp rx=0 tx=0).stop 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 shutdown_connections 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4e7c035690 0x7f4e7c037b50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 --2- 172.21.15.96:0/2303040061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e9809c960 0x7f4e981cc7f0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 >> 172.21.15.96:0/2303040061 conn(0x7f4e9806e020 msgr2=0x7f4e9812e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 shutdown_connections 2024-09-13T17:44:32.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.535+0000 7f4ea0437640 1 -- 172.21.15.96:0/2303040061 wait complete. 2024-09-13T17:44:32.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:32.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.667+0000 7fce8282b640 1 Processor -- start 2024-09-13T17:44:32.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.667+0000 7fce8282b640 1 -- start start 2024-09-13T17:44:32.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce7c09e820 0x7fce7c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0a68b0 con 0x7fce7c134e70 2024-09-13T17:44:32.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c09ddc0 con 0x7fce7c09e820 2024-09-13T17:44:32.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fce7c09e820 0x7fce7c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fce7c134e70 0x7fce7c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54516/0 (socket says 172.21.15.96:54516) 2024-09-13T17:44:32.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7bfff640 1 -- 172.21.15.96:0/624398830 learned_addr learned my addr 172.21.15.96:0/624398830 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:32.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7b7fe640 1 -- 172.21.15.96:0/624398830 >> v1:172.21.15.96:6789/0 conn(0x7fce7c134e70 legacy=0x7fce7c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7b7fe640 1 -- 172.21.15.96:0/624398830 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c136970 con 0x7fce7c09e820 2024-09-13T17:44:32.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7b7fe640 1 --2- 172.21.15.96:0/624398830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c13ae50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce64009740 tx=0x7fce6401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e25f89b454fe15 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7affd640 1 -- 172.21.15.96:0/624398830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce640ab530 con 0x7fce7c09e820 2024-09-13T17:44:32.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7affd640 1 -- 172.21.15.96:0/624398830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce640a7050 con 0x7fce7c09e820 2024-09-13T17:44:32.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce7affd640 1 -- 172.21.15.96:0/624398830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce640ab7e0 con 0x7fce7c09e820 2024-09-13T17:44:32.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- 172.21.15.96:0/624398830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 msgr2=0x7fce7c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 --2- 172.21.15.96:0/624398830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c13ae50 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fce64009740 tx=0x7fce6401c420 comp rx=0 tx=0).stop 2024-09-13T17:44:32.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- 172.21.15.96:0/624398830 shutdown_connections 2024-09-13T17:44:32.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 --2- 172.21.15.96:0/624398830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c13ae50 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- 172.21.15.96:0/624398830 >> 172.21.15.96:0/624398830 conn(0x7fce7c098a60 msgr2=0x7fce7c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- 172.21.15.96:0/624398830 shutdown_connections 2024-09-13T17:44:32.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.671+0000 7fce8282b640 1 -- 172.21.15.96:0/624398830 wait complete. 2024-09-13T17:44:32.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 Processor -- start 2024-09-13T17:44:32.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 -- start start 2024-09-13T17:44:32.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0a3070 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55462/0 (socket says 172.21.15.96:55462) 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7bfff640 1 -- 172.21.15.96:0/2786322653 learned_addr learned my addr 172.21.15.96:0/2786322653 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7bfff640 1 -- 172.21.15.96:0/2786322653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64009420 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7bfff640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce7400ca90 tx=0x7fce7400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce74004ca0 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74004e00 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c0a2e20 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fce7400aba0 con 0x7fce7c09e820 2024-09-13T17:44:32.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c0a17b0 con 0x7fce7c09e820 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fce7400ad00 con 0x7fce7c09e820 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c0a31c0 con 0x7fce7c09e820 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 0x7fce5c037b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7fce74047d50 con 0x7fce7c09e820 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7b7fe640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 0x7fce5c037b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.675+0000 7fce7b7fe640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 0x7fce5c037b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce640ad000 tx=0x7fce64008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.679+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fce74010710 con 0x7fce7c09e820 2024-09-13T17:44:32.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce7c06a1b0 con 0x7fce7c09e820 2024-09-13T17:44:32.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce78ff9640 1 -- 172.21.15.96:0/2786322653 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3220 (secure 0 0 0) 0x7fce74018030 con 0x7fce7c09e820 2024-09-13T17:44:32.864 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:32.864 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/1651635905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1651635905},{"type":"v1","addr":"172.21.15.96:6829","nonce":1651635905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:43:56.008006+0000","modified":"2024-09-13T17:43:59.019423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4740},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4740":{"gid":4740,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":7,"addr":"172.21.15.96:6827/285199718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":285199718},{"type":"v1","addr":"172.21.15.96:6827","nonce":285199718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs_b","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-09-13T17:44:32.865 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-09-13T17:44:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 msgr2=0x7fce5c037b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 0x7fce5c037b00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fce640ad000 tx=0x7fce64008040 comp rx=0 tx=0).stop 2024-09-13T17:44:32.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 msgr2=0x7fce7c1cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fce7400ca90 tx=0x7fce7400cf50 comp rx=0 tx=0).stop 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 shutdown_connections 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fce5c035640 0x7fce5c037b00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 --2- 172.21.15.96:0/2786322653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce7c09e820 0x7fce7c1cda00 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 >> 172.21.15.96:0/2786322653 conn(0x7fce7c098a60 msgr2=0x7fce7c134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 shutdown_connections 2024-09-13T17:44:32.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.859+0000 7fce8282b640 1 -- 172.21.15.96:0/2786322653 wait complete. 2024-09-13T17:44:32.882 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:32.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b3a01c640 1 Processor -- start 2024-09-13T17:44:32.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b3a01c640 1 -- start start 2024-09-13T17:44:32.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b3a01c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2b3406f6d0 0x7f2b34134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:32.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b3a01c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b3406e130 con 0x7f2b3409dc20 2024-09-13T17:44:32.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b3a01c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340a6750 con 0x7f2b3406f6d0 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b337fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2b3406f6d0 0x7f2b34134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b2bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2b3409dc20 0x7f2b34134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54530/0 (socket says 172.21.15.96:54530) 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b2bfff640 1 -- 172.21.15.96:0/1693763297 learned_addr learned my addr 172.21.15.96:0/1693763297 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b337fe640 1 -- 172.21.15.96:0/1693763297 >> v1:172.21.15.96:6789/0 conn(0x7f2b3409dc20 legacy=0x7f2b34134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b32ffd640 1 -- 172.21.15.96:0/1693763297 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658193460 0 0) 0x7f2b3406e130 con 0x7f2b3409dc20 2024-09-13T17:44:32.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.991+0000 7f2b337fe640 1 -- 172.21.15.96:0/1693763297 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340712e0 con 0x7f2b3406f6d0 2024-09-13T17:44:32.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 --2- 172.21.15.96:0/1693763297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b34134ab0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b9e0 tx=0x7f2b2401ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a15e9dde102e2a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:32.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b32ffd640 1 -- 172.21.15.96:0/1693763297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b240a7750 con 0x7f2b3406f6d0 2024-09-13T17:44:33.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b32ffd640 1 -- 172.21.15.96:0/1693763297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b240a78b0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b32ffd640 1 -- 172.21.15.96:0/1693763297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b240a7b70 con 0x7f2b3406f6d0 2024-09-13T17:44:33.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- 172.21.15.96:0/1693763297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 msgr2=0x7f2b34134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/1693763297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b34134ab0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b9e0 tx=0x7f2b2401ea60 comp rx=0 tx=0).stop 2024-09-13T17:44:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- 172.21.15.96:0/1693763297 shutdown_connections 2024-09-13T17:44:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/1693763297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b34134ab0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- 172.21.15.96:0/1693763297 >> 172.21.15.96:0/1693763297 conn(0x7f2b34098a60 msgr2=0x7f2b3409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- 172.21.15.96:0/1693763297 shutdown_connections 2024-09-13T17:44:33.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- 172.21.15.96:0/1693763297 wait complete. 2024-09-13T17:44:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 Processor -- start 2024-09-13T17:44:33.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- start start 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b3a01c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341233d0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55472/0 (socket says 172.21.15.96:55472) 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 -- 172.21.15.96:0/945211096 learned_addr learned my addr 172.21.15.96:0/945211096 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 -- 172.21.15.96:0/945211096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2400b6c0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b337fe640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b2401c690 tx=0x7f2b2401b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b240be070 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.995+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b240af8c0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b341221d0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b240afb80 con 0x7f2b3406f6d0 2024-09-13T17:44:33.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3409b160 con 0x7f2b3406f6d0 2024-09-13T17:44:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f2b240b6070 con 0x7f2b3406f6d0 2024-09-13T17:44:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3409ac10 con 0x7f2b3406f6d0 2024-09-13T17:44:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b30ff9640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 0x7f2b14037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f2b240e2740 con 0x7f2b3406f6d0 2024-09-13T17:44:33.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b2bfff640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 0x7f2b14037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:32.999+0000 7f2b2bfff640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 0x7f2b14037b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c007fd0 tx=0x7f2b1c007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.003+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2b240ad340 con 0x7f2b3406f6d0 2024-09-13T17:44:33.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.163+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b3406f2e0 con 0x7f2b3406f6d0 2024-09-13T17:44:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b30ff9640 1 -- 172.21.15.96:0/945211096 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+11627 (secure 0 0 0) 0x7f2b240b4020 con 0x7f2b3406f6d0 2024-09-13T17:44:33.171 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:33.171 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":46,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:30.289043+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_b_metadata","create_time":"2024-09-13T17:43:54.524016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_b_data","create_time":"2024-09-13T17:43:54.975952+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs_b"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6829/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6828/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 msgr2=0x7f2b14037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 0x7f2b14037b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c007fd0 tx=0x7f2b1c007920 comp rx=0 tx=0).stop 2024-09-13T17:44:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 msgr2=0x7f2b340a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f2b2401c690 tx=0x7f2b2401b800 comp rx=0 tx=0).stop 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 shutdown_connections 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2b14035690 0x7f2b14037b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 --2- 172.21.15.96:0/945211096 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b3406f6d0 0x7f2b340a3f70 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 >> 172.21.15.96:0/945211096 conn(0x7f2b34098a60 msgr2=0x7f2b34137ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 shutdown_connections 2024-09-13T17:44:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.167+0000 7f2b3a01c640 1 -- 172.21.15.96:0/945211096 wait complete. 2024-09-13T17:44:33.198 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs_b 2024-09-13T17:44:33.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9c9dd640 1 Processor -- start 2024-09-13T17:44:33.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9c9dd640 1 -- start start 2024-09-13T17:44:33.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9c9dd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5f9809e820 0x7f5f9813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9c9dd640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f980a68b0 con 0x7f5f98134e70 2024-09-13T17:44:33.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9c9dd640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f9809ddc0 con 0x7f5f9809e820 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5f98134e70 0x7f5f98135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54538/0 (socket says 172.21.15.96:54538) 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9659c640 1 -- 172.21.15.96:0/3549297997 learned_addr learned my addr 172.21.15.96:0/3549297997 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f95d9b640 1 --2- 172.21.15.96:0/3549297997 >> v2:172.21.15.96:3300/0 conn(0x7f5f9809e820 0x7f5f9813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9559a640 1 -- 172.21.15.96:0/3549297997 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856529783 0 0) 0x7f5f980a68b0 con 0x7f5f98134e70 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9559a640 1 -- 172.21.15.96:0/3549297997 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f80003530 con 0x7f5f98134e70 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f95d9b640 1 -- 172.21.15.96:0/3549297997 >> v1:172.21.15.96:6789/0 conn(0x7f5f98134e70 legacy=0x7f5f98135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f95d9b640 1 -- 172.21.15.96:0/3549297997 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f98136970 con 0x7f5f9809e820 2024-09-13T17:44:33.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f95d9b640 1 --2- 172.21.15.96:0/3549297997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f9813ae50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009740 tx=0x7f5f8c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3aba03d42ca280 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9559a640 1 -- 172.21.15.96:0/3549297997 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c0ab530 con 0x7f5f9809e820 2024-09-13T17:44:33.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.315+0000 7f5f9559a640 1 -- 172.21.15.96:0/3549297997 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c0a7050 con 0x7f5f9809e820 2024-09-13T17:44:33.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9559a640 1 -- 172.21.15.96:0/3549297997 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c0ab7e0 con 0x7f5f9809e820 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/3549297997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 msgr2=0x7f5f9813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/3549297997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f9813ae50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009740 tx=0x7f5f8c01c420 comp rx=0 tx=0).stop 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/3549297997 shutdown_connections 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/3549297997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f9813ae50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/3549297997 >> 172.21.15.96:0/3549297997 conn(0x7f5f98098a60 msgr2=0x7f5f9809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/3549297997 shutdown_connections 2024-09-13T17:44:33.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/3549297997 wait complete. 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 Processor -- start 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- start start 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f98071730 con 0x7f5f9809e820 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55484/0 (socket says 172.21.15.96:55484) 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9659c640 1 -- 172.21.15.96:0/756124223 learned_addr learned my addr 172.21.15.96:0/756124223 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9659c640 1 -- 172.21.15.96:0/756124223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c009420 con 0x7f5f9809e820 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9659c640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800c450 tx=0x7f5f8800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f88005c20 con 0x7f5f9809e820 2024-09-13T17:44:33.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88005d80 con 0x7f5f9809e820 2024-09-13T17:44:33.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f980a3b80 con 0x7f5f9809e820 2024-09-13T17:44:33.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f880071a0 con 0x7f5f9809e820 2024-09-13T17:44:33.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f980a3e90 con 0x7f5f9809e820 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.319+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9813a430 con 0x7f5f9809e820 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.323+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f5f88007300 con 0x7f5f9809e820 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.323+0000 7f5f86ffd640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 0x7f5f78037b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.323+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4221+0+0 (secure 0 0 0) 0x7f5f88047870 con 0x7f5f9809e820 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.323+0000 7f5f95d9b640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 0x7f5f78037b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.323+0000 7f5f95d9b640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 0x7f5f78037b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0ad000 tx=0x7f5f8c008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.327+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5f880196e0 con 0x7f5f9809e820 2024-09-13T17:44:33.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.519+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs_b"} v 0) v1 -- 0x7f5f9806a1b0 con 0x7f5f9809e820 2024-09-13T17:44:33.525 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:44:33.519+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:44:33.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f86ffd640 1 -- 172.21.15.96:0/756124223 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs_b"}]=0 cephfs_b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 165+0+0 (secure 0 0 0) 0x7f5f88017020 con 0x7f5f9809e820 2024-09-13T17:44:33.535 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs_b marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 msgr2=0x7f5f78037b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 0x7f5f78037b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0ad000 tx=0x7f5f8c008040 comp rx=0 tx=0).stop 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 msgr2=0x7f5f981d3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800c450 tx=0x7f5f8800c910 comp rx=0 tx=0).stop 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 shutdown_connections 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5f78035640 0x7f5f78037b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 --2- 172.21.15.96:0/756124223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5f9809e820 0x7f5f981d3eb0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 >> 172.21.15.96:0/756124223 conn(0x7f5f98098a60 msgr2=0x7f5f98133dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:33.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 shutdown_connections 2024-09-13T17:44:33.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.531+0000 7f5f9c9dd640 1 -- 172.21.15.96:0/756124223 wait complete. 2024-09-13T17:44:33.539 INFO:tasks.ceph.mds.a.smithi096.stderr: -2103> 2024-09-13T17:43:57.608+0000 7f53071ac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:44:33.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs_b --yes-i-really-mean-it 2024-09-13T17:44:33.582 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:44:33.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.675+0000 7f634c352640 1 Processor -- start 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.675+0000 7f634c352640 1 -- start start 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.675+0000 7f634c352640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6344065c80 0x7f63441318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.675+0000 7f634c352640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634406f500 con 0x7f6344070400 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.675+0000 7f634c352640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634406f810 con 0x7f6344065c80 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634a0ee640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6344065c80 0x7f63441318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f63498ed640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6344070400 0x7f6344131de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54552/0 (socket says 172.21.15.96:54552) 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f63498ed640 1 -- 172.21.15.96:0/13459873 learned_addr learned my addr 172.21.15.96:0/13459873 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:33.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634a0ee640 1 -- 172.21.15.96:0/13459873 >> v1:172.21.15.96:6789/0 conn(0x7f6344070400 legacy=0x7f6344131de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634a0ee640 1 -- 172.21.15.96:0/13459873 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63440a28b0 con 0x7f6344065c80 2024-09-13T17:44:33.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634a0ee640 1 --2- 172.21.15.96:0/13459873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f63441318a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6340009740 tx=0x7f634001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49874da4fdd932b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f63490ec640 1 -- 172.21.15.96:0/13459873 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63400ab530 con 0x7f6344065c80 2024-09-13T17:44:33.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f63490ec640 1 -- 172.21.15.96:0/13459873 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63400a7050 con 0x7f6344065c80 2024-09-13T17:44:33.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f63490ec640 1 -- 172.21.15.96:0/13459873 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63400ab7e0 con 0x7f6344065c80 2024-09-13T17:44:33.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 -- 172.21.15.96:0/13459873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 msgr2=0x7f63441318a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:33.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 --2- 172.21.15.96:0/13459873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f63441318a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6340009740 tx=0x7f634001c390 comp rx=0 tx=0).stop 2024-09-13T17:44:33.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 -- 172.21.15.96:0/13459873 shutdown_connections 2024-09-13T17:44:33.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 --2- 172.21.15.96:0/13459873 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f63441318a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:33.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 -- 172.21.15.96:0/13459873 >> 172.21.15.96:0/13459873 conn(0x7f634406a150 msgr2=0x7f634406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:33.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 -- 172.21.15.96:0/13459873 shutdown_connections 2024-09-13T17:44:33.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.679+0000 7f634c352640 1 -- 172.21.15.96:0/13459873 wait complete. 2024-09-13T17:44:33.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 Processor -- start 2024-09-13T17:44:33.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 -- start start 2024-09-13T17:44:33.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634409dfa0 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634a0ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634a0ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55510/0 (socket says 172.21.15.96:55510) 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634a0ee640 1 -- 172.21.15.96:0/303304795 learned_addr learned my addr 172.21.15.96:0/303304795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634a0ee640 1 -- 172.21.15.96:0/303304795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6340009420 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634a0ee640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6340019fc0 tx=0x7f634001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63400b7450 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63400b75b0 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63440a08f0 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63400b7870 con 0x7f6344065c80 2024-09-13T17:44:33.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634409f1b0 con 0x7f6344065c80 2024-09-13T17:44:33.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f63400b79d0 con 0x7f6344065c80 2024-09-13T17:44:33.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 0x7f6328037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:33.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4307+0+0 (secure 0 0 0) 0x7f63400e2780 con 0x7f6344065c80 2024-09-13T17:44:33.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63440a1b60 con 0x7f6344065c80 2024-09-13T17:44:33.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f63498ed640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 0x7f6328037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:33.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.683+0000 7f63498ed640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 0x7f6328037b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6344070810 tx=0x7f6334007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:33.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.687+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f63400bb020 con 0x7f6344065c80 2024-09-13T17:44:33.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:33.871+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs_b", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f63440668c0 con 0x7f6344065c80 2024-09-13T17:44:34.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.539+0000 7f633affd640 1 -- 172.21.15.96:0/303304795 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs_b", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 106+0+0 (secure 0 0 0) 0x7f63400b0070 con 0x7f6344065c80 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.539+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 msgr2=0x7f6328037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 0x7f6328037b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6344070810 tx=0x7f6334007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 msgr2=0x7f634409fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6340019fc0 tx=0x7f634001abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 shutdown_connections 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f6328035690 0x7f6328037b50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 --2- 172.21.15.96:0/303304795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6344065c80 0x7f634409fbf0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 >> 172.21.15.96:0/303304795 conn(0x7f634406a150 msgr2=0x7f63440a3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:34.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 shutdown_connections 2024-09-13T17:44:34.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.543+0000 7f634c352640 1 -- 172.21.15.96:0/303304795 wait complete. 2024-09-13T17:44:34.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_b_metadata cephfs_b_metadata --yes-i-really-really-mean-it 2024-09-13T17:44:34.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f89a1b640 1 Processor -- start 2024-09-13T17:44:34.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f89a1b640 1 -- start start 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f89a1b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f8409fd20 0x7f3f8413d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f89a1b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8409e520 con 0x7f3f84138290 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f89a1b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f840aabe0 con 0x7f3f8409fd20 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f827fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f8409fd20 0x7f3f8413d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f827fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f8409fd20 0x7f3f8413d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55526/0 (socket says 172.21.15.96:55526) 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f827fc640 1 -- 172.21.15.96:0/4233451025 learned_addr learned my addr 172.21.15.96:0/4233451025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f827fc640 1 -- 172.21.15.96:0/4233451025 >> v1:172.21.15.96:6789/0 conn(0x7f3f84138290 legacy=0x7f3f84138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:34.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.731+0000 7f3f827fc640 1 -- 172.21.15.96:0/4233451025 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8409d610 con 0x7f3f8409fd20 2024-09-13T17:44:34.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f827fc640 1 --2- 172.21.15.96:0/4233451025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f8413d1d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097f0 tx=0x7f3f7801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7864bcba59ecaf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:34.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f81ffb640 1 -- 172.21.15.96:0/4233451025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f780ad460 con 0x7f3f8409fd20 2024-09-13T17:44:34.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f81ffb640 1 -- 172.21.15.96:0/4233451025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f780ad5c0 con 0x7f3f8409fd20 2024-09-13T17:44:34.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f81ffb640 1 -- 172.21.15.96:0/4233451025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f780ad880 con 0x7f3f8409fd20 2024-09-13T17:44:34.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- 172.21.15.96:0/4233451025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 msgr2=0x7f3f8413d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:34.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/4233451025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f8413d1d0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f3f780097f0 tx=0x7f3f7801c450 comp rx=0 tx=0).stop 2024-09-13T17:44:34.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- 172.21.15.96:0/4233451025 shutdown_connections 2024-09-13T17:44:34.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/4233451025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f8413d1d0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:34.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- 172.21.15.96:0/4233451025 >> 172.21.15.96:0/4233451025 conn(0x7f3f8406c060 msgr2=0x7f3f8406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:34.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- 172.21.15.96:0/4233451025 shutdown_connections 2024-09-13T17:44:34.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- 172.21.15.96:0/4233451025 wait complete. 2024-09-13T17:44:34.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 Processor -- start 2024-09-13T17:44:34.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- start start 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f89a1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841cbec0 con 0x7f3f8409fd20 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f82ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55538/0 (socket says 172.21.15.96:55538) 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f82ffd640 1 -- 172.21.15.96:0/60727979 learned_addr learned my addr 172.21.15.96:0/60727979 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f82ffd640 1 -- 172.21.15.96:0/60727979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f780094d0 con 0x7f3f8409fd20 2024-09-13T17:44:34.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.735+0000 7f3f82ffd640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c7f0 tx=0x7f3f7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c006c90 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7c006e10 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8409e1a0 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f7c00a960 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f840a3fd0 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f3f7c00aac0 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f840a5640 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 0x7f3f70037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4307+0+0 (secure 0 0 0) 0x7f3f7c012c70 con 0x7f3f8409fd20 2024-09-13T17:44:34.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f827fc640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 0x7f3f70037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:34.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.739+0000 7f3f827fc640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 0x7f3f70037b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f780af000 tx=0x7f3f780a7000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:34.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.743+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f7c019310 con 0x7f3f8409fd20 2024-09-13T17:44:34.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:34.903+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_b_metadata", "pool2": "cephfs_b_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f84065e90 con 0x7f3f8409fd20 2024-09-13T17:44:35.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.543+0000 7f3f6bfff640 1 -- 172.21.15.96:0/60727979 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_b_metadata", "pool2": "cephfs_b_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_b_metadata' removed v48) v1 ==== 187+0+0 (secure 0 0 0) 0x7f3f7c016030 con 0x7f3f8409fd20 2024-09-13T17:44:35.551 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_b_metadata' removed 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 msgr2=0x7f3f70037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 0x7f3f70037b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f780af000 tx=0x7f3f780a7000 comp rx=0 tx=0).stop 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 msgr2=0x7f3f841cc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00c7f0 tx=0x7f3f7c00ccb0 comp rx=0 tx=0).stop 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 shutdown_connections 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f70035690 0x7f3f70037b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 --2- 172.21.15.96:0/60727979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f8409fd20 0x7f3f841cc0b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:35.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 >> 172.21.15.96:0/60727979 conn(0x7f3f8406c060 msgr2=0x7f3f841358b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:35.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 shutdown_connections 2024-09-13T17:44:35.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.551+0000 7f3f89a1b640 1 -- 172.21.15.96:0/60727979 wait complete. 2024-09-13T17:44:35.578 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_b_data cephfs_b_data --yes-i-really-really-mean-it 2024-09-13T17:44:35.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3f652640 1 Processor -- start 2024-09-13T17:44:35.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3f652640 1 -- start start 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3f652640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fee3809e890 0x7fee3813ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3f652640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3809e0d0 con 0x7fee38135e80 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3f652640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3809ef50 con 0x7fee3809e890 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3d3ee640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fee38135e80 0x7fee38136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54570/0 (socket says 172.21.15.96:54570) 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3cbed640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fee3809e890 0x7fee3813ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3d3ee640 1 -- 172.21.15.96:0/1550972126 learned_addr learned my addr 172.21.15.96:0/1550972126 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee2ffff640 1 -- 172.21.15.96:0/1550972126 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006090133 0 0) 0x7fee3809e0d0 con 0x7fee38135e80 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee2ffff640 1 -- 172.21.15.96:0/1550972126 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee24003530 con 0x7fee38135e80 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3cbed640 1 -- 172.21.15.96:0/1550972126 >> v1:172.21.15.96:6789/0 conn(0x7fee38135e80 legacy=0x7fee38136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:35.702 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.695+0000 7fee3cbed640 1 -- 172.21.15.96:0/1550972126 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3809bf40 con 0x7fee3809e890 2024-09-13T17:44:35.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3cbed640 1 --2- 172.21.15.96:0/1550972126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3813ade0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee28009740 tx=0x7fee2801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d777fd3a2f478feb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:35.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee2ffff640 1 -- 172.21.15.96:0/1550972126 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee280ab550 con 0x7fee3809e890 2024-09-13T17:44:35.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee2ffff640 1 -- 172.21.15.96:0/1550972126 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee280ab6b0 con 0x7fee3809e890 2024-09-13T17:44:35.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee2ffff640 1 -- 172.21.15.96:0/1550972126 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee280ab970 con 0x7fee3809e890 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- 172.21.15.96:0/1550972126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 msgr2=0x7fee3813ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 --2- 172.21.15.96:0/1550972126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3813ade0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee28009740 tx=0x7fee2801c420 comp rx=0 tx=0).stop 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- 172.21.15.96:0/1550972126 shutdown_connections 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 --2- 172.21.15.96:0/1550972126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3813ade0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- 172.21.15.96:0/1550972126 >> 172.21.15.96:0/1550972126 conn(0x7fee38098a60 msgr2=0x7fee3809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- 172.21.15.96:0/1550972126 shutdown_connections 2024-09-13T17:44:35.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- 172.21.15.96:0/1550972126 wait complete. 2024-09-13T17:44:35.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 Processor -- start 2024-09-13T17:44:35.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- start start 2024-09-13T17:44:35.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:35.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.699+0000 7fee3f652640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3811f3f0 con 0x7fee3809e890 2024-09-13T17:44:35.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3d3ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:35.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3d3ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55556/0 (socket says 172.21.15.96:55556) 2024-09-13T17:44:35.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3d3ee640 1 -- 172.21.15.96:0/1439001162 learned_addr learned my addr 172.21.15.96:0/1439001162 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3d3ee640 1 -- 172.21.15.96:0/1439001162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee28009420 con 0x7fee3809e890 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3d3ee640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fee3400c450 tx=0x7fee3400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee34005c20 con 0x7fee3809e890 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee34005d80 con 0x7fee3809e890 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3811d4e0 con 0x7fee3809e890 2024-09-13T17:44:35.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee340071a0 con 0x7fee3809e890 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3811d780 con 0x7fee3809e890 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fee34007300 con 0x7fee3809e890 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3811e180 con 0x7fee3809e890 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 0x7fee1c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 3865+0+0 (secure 0 0 0) 0x7fee34047150 con 0x7fee3809e890 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3cbed640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 0x7fee1c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:35.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.703+0000 7fee3cbed640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 0x7fee1c037b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fee280af000 tx=0x7fee28008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:35.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.707+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fee34012400 con 0x7fee3809e890 2024-09-13T17:44:35.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:35.867+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_b_data", "pool2": "cephfs_b_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fee3806ee50 con 0x7fee3809e890 2024-09-13T17:44:36.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.563+0000 7fee2dffb640 1 -- 172.21.15.96:0/1439001162 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_b_data", "pool2": "cephfs_b_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_b_data' removed v49) v1 ==== 175+0+0 (secure 0 0 0) 0x7fee34017020 con 0x7fee3809e890 2024-09-13T17:44:36.572 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_b_data' removed 2024-09-13T17:44:36.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.571+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 msgr2=0x7fee1c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.571+0000 7fee3f652640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 0x7fee1c037b50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fee280af000 tx=0x7fee28008040 comp rx=0 tx=0).stop 2024-09-13T17:44:36.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.571+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 msgr2=0x7fee3811eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.571+0000 7fee3f652640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fee3400c450 tx=0x7fee3400c910 comp rx=0 tx=0).stop 2024-09-13T17:44:36.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 shutdown_connections 2024-09-13T17:44:36.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fee1c035690 0x7fee1c037b50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:36.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 --2- 172.21.15.96:0/1439001162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3809e890 0x7fee3811eeb0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:36.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 >> 172.21.15.96:0/1439001162 conn(0x7fee38098a60 msgr2=0x7fee38134fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:36.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 shutdown_connections 2024-09-13T17:44:36.580 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.575+0000 7fee3f652640 1 -- 172.21.15.96:0/1439001162 wait complete. 2024-09-13T17:44:36.603 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:44:36.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f207b1a4640 1 Processor -- start 2024-09-13T17:44:36.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f207b1a4640 1 -- start start 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f207b1a4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2074137490 0x7f2074139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f207b1a4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2074071670 con 0x7f2074134670 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f207b1a4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2074071360 con 0x7f2074137490 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f2073fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2074137490 0x7f2074139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f2078f40640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2074134670 0x7f2074134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54574/0 (socket says 172.21.15.96:54574) 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f2078f40640 1 -- 172.21.15.96:0/1182782595 learned_addr learned my addr 172.21.15.96:0/1182782595 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f2073fff640 1 -- 172.21.15.96:0/1182782595 >> v1:172.21.15.96:6789/0 conn(0x7f2074134670 legacy=0x7f2074134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.723+0000 7f2073fff640 1 -- 172.21.15.96:0/1182782595 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740a7200 con 0x7f2074137490 2024-09-13T17:44:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f2073fff640 1 --2- 172.21.15.96:0/1182782595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074137490 0x7f2074139880 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f206800a8e0 tx=0x7f206801d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d631afe9d76503d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f20737fe640 1 -- 172.21.15.96:0/1182782595 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20680a9550 con 0x7f2074137490 2024-09-13T17:44:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f20737fe640 1 -- 172.21.15.96:0/1182782595 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20680a96b0 con 0x7f2074137490 2024-09-13T17:44:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f20737fe640 1 -- 172.21.15.96:0/1182782595 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20680a9970 con 0x7f2074137490 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- 172.21.15.96:0/1182782595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074137490 msgr2=0x7f2074139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 --2- 172.21.15.96:0/1182782595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074137490 0x7f2074139880 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f206800a8e0 tx=0x7f206801d760 comp rx=0 tx=0).stop 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- 172.21.15.96:0/1182782595 shutdown_connections 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 --2- 172.21.15.96:0/1182782595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074137490 0x7f2074139880 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- 172.21.15.96:0/1182782595 >> 172.21.15.96:0/1182782595 conn(0x7f2074098a60 msgr2=0x7f207409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- 172.21.15.96:0/1182782595 shutdown_connections 2024-09-13T17:44:36.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- 172.21.15.96:0/1182782595 wait complete. 2024-09-13T17:44:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 Processor -- start 2024-09-13T17:44:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.727+0000 7f207b1a4640 1 -- start start 2024-09-13T17:44:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f207b1a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:36.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f207b1a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740a3df0 con 0x7f2074134670 2024-09-13T17:44:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2078f40640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2078f40640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55574/0 (socket says 172.21.15.96:55574) 2024-09-13T17:44:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2078f40640 1 -- 172.21.15.96:0/3108638627 learned_addr learned my addr 172.21.15.96:0/3108638627 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2078f40640 1 -- 172.21.15.96:0/3108638627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206800a5c0 con 0x7f2074134670 2024-09-13T17:44:36.736 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2078f40640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f206400c480 tx=0x7f206400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2064005c20 con 0x7f2074134670 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064005d80 con 0x7f2074134670 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20640071a0 con 0x7f2074134670 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2074136d80 con 0x7f2074134670 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20740a5b90 con 0x7f2074134670 2024-09-13T17:44:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20740a3fc0 con 0x7f2074134670 2024-09-13T17:44:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f2064007300 con 0x7f2074134670 2024-09-13T17:44:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 0x7f2060037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f2064046f90 con 0x7f2074134670 2024-09-13T17:44:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2073fff640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 0x7f2060037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:36.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.731+0000 7f2073fff640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 0x7f2060037b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20680b0000 tx=0x7f20680a8000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:36.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.735+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f206401b3b0 con 0x7f2074134670 2024-09-13T17:44:36.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.895+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2074134a50 con 0x7f2074134670 2024-09-13T17:44:36.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.899+0000 7f20717fa640 1 -- 172.21.15.96:0/3108638627 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2064010340 con 0x7f2074134670 2024-09-13T17:44:36.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 msgr2=0x7f2060037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 0x7f2060037b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f20680b0000 tx=0x7f20680a8000 comp rx=0 tx=0).stop 2024-09-13T17:44:36.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 msgr2=0x7f20740a2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f206400c480 tx=0x7f206400c940 comp rx=0 tx=0).stop 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 shutdown_connections 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2060035690 0x7f2060037b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 --2- 172.21.15.96:0/3108638627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2074134670 0x7f20740a2160 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 >> 172.21.15.96:0/3108638627 conn(0x7f2074098a60 msgr2=0x7f2074138f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 shutdown_connections 2024-09-13T17:44:36.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:36.903+0000 7f207b1a4640 1 -- 172.21.15.96:0/3108638627 wait complete. 2024-09-13T17:44:36.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs' 2024-09-13T17:44:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f11a557c640 1 Processor -- start 2024-09-13T17:44:37.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f11a557c640 1 -- start start 2024-09-13T17:44:37.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f11a557c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11a0134e70 0x7f11a009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f11a557c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a009ed60 con 0x7f11a0135250 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f11a557c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a009ddc0 con 0x7f11a0134e70 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11a0134e70 0x7f11a009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119e7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f11a0135250 0x7f11a0135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54582/0 (socket says 172.21.15.96:54582) 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119e7fc640 1 -- 172.21.15.96:0/1397392150 learned_addr learned my addr 172.21.15.96:0/1397392150 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119effd640 1 -- 172.21.15.96:0/1397392150 >> v1:172.21.15.96:6789/0 conn(0x7f11a0135250 legacy=0x7f11a0135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119effd640 1 -- 172.21.15.96:0/1397392150 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a00a68b0 con 0x7f11a0134e70 2024-09-13T17:44:37.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.039+0000 7f119effd640 1 --2- 172.21.15.96:0/1397392150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a009e820 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1190009740 tx=0x7f119001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1cdd3e23910c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119dffb640 1 -- 172.21.15.96:0/1397392150 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900ab530 con 0x7f11a0134e70 2024-09-13T17:44:37.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119dffb640 1 -- 172.21.15.96:0/1397392150 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11900a7050 con 0x7f11a0134e70 2024-09-13T17:44:37.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119dffb640 1 -- 172.21.15.96:0/1397392150 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900ab7e0 con 0x7f11a0134e70 2024-09-13T17:44:37.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/1397392150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 msgr2=0x7f11a009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 --2- 172.21.15.96:0/1397392150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a009e820 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f1190009740 tx=0x7f119001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:37.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/1397392150 shutdown_connections 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 --2- 172.21.15.96:0/1397392150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a009e820 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/1397392150 >> 172.21.15.96:0/1397392150 conn(0x7f11a0098a60 msgr2=0x7f11a009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/1397392150 shutdown_connections 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/1397392150 wait complete. 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 Processor -- start 2024-09-13T17:44:37.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- start start 2024-09-13T17:44:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a011c930 con 0x7f11a0134e70 2024-09-13T17:44:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55588/0 (socket says 172.21.15.96:55588) 2024-09-13T17:44:37.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119effd640 1 -- 172.21.15.96:0/3005361220 learned_addr learned my addr 172.21.15.96:0/3005361220 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:37.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119effd640 1 -- 172.21.15.96:0/3005361220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1190009420 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f119effd640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f119001a050 tx=0x7f11900191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900abea0 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11900b4410 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a011cb60 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11900b46b0 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.043+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a00a5120 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f11900b2040 con 0x7f11a0134e70 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f11877fe640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 0x7f118c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a013b4b0 con 0x7f11a0134e70 2024-09-13T17:44:37.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f11900e1c90 con 0x7f11a0134e70 2024-09-13T17:44:37.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f119e7fc640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 0x7f118c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.047+0000 7f119e7fc640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 0x7f118c037b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1194001610 tx=0x7f1194007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.051+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f11900b0020 con 0x7f11a0134e70 2024-09-13T17:44:37.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.267+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs"]} v 0) v1 -- 0x7f11a013af00 con 0x7f11a0134e70 2024-09-13T17:44:37.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.559+0000 7f11877fe640 1 -- 172.21.15.96:0/3005361220 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_dumpfs"]}]=0 v162) v1 ==== 124+0+0 (secure 0 0 0) 0x7f11900b4810 con 0x7f11a0134e70 2024-09-13T17:44:37.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 msgr2=0x7f118c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 0x7f118c037b50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1194001610 tx=0x7f1194007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 msgr2=0x7f11a006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f119001a050 tx=0x7f11900191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 shutdown_connections 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f118c035690 0x7f118c037b50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 --2- 172.21.15.96:0/3005361220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11a0134e70 0x7f11a006f6d0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 >> 172.21.15.96:0/3005361220 conn(0x7f11a0098a60 msgr2=0x7f11a0133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 shutdown_connections 2024-09-13T17:44:37.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.563+0000 7f11a557c640 1 -- 172.21.15.96:0/3005361220 wait complete. 2024-09-13T17:44:37.592 INFO:tasks.cephfs_test_runner:Tests 'cephfs-top --dumpfs' output is valid ... ok 2024-09-13T17:44:37.592 INFO:tasks.cephfs_test_runner:Starting test: test_fstop (tasks.cephfs.test_fstop.TestFSTop) 2024-09-13T17:44:37.592 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:37.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.711+0000 7f8dc8341640 1 Processor -- start 2024-09-13T17:44:37.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.711+0000 7f8dc8341640 1 -- start start 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.711+0000 7f8dc8341640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8dc0134e70 0x7f8dc009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.711+0000 7f8dc8341640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc009ed60 con 0x7f8dc0135250 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.711+0000 7f8dc8341640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc009ddc0 con 0x7f8dc0134e70 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc60dd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8dc0134e70 0x7f8dc009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc58dc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8dc0135250 0x7f8dc0135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54592/0 (socket says 172.21.15.96:54592) 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc58dc640 1 -- 172.21.15.96:0/758374314 learned_addr learned my addr 172.21.15.96:0/758374314 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc60dd640 1 -- 172.21.15.96:0/758374314 >> v1:172.21.15.96:6789/0 conn(0x7f8dc0135250 legacy=0x7f8dc0135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc60dd640 1 -- 172.21.15.96:0/758374314 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc00a68b0 con 0x7f8dc0134e70 2024-09-13T17:44:37.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc60dd640 1 --2- 172.21.15.96:0/758374314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc009e820 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009d90 tx=0x7f8db801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1d8e3996564c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc50db640 1 -- 172.21.15.96:0/758374314 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8db80a57b0 con 0x7f8dc0134e70 2024-09-13T17:44:37.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc50db640 1 -- 172.21.15.96:0/758374314 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db80a5910 con 0x7f8dc0134e70 2024-09-13T17:44:37.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc50db640 1 -- 172.21.15.96:0/758374314 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8db80a5bd0 con 0x7f8dc0134e70 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 -- 172.21.15.96:0/758374314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 msgr2=0x7f8dc009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 --2- 172.21.15.96:0/758374314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc009e820 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009d90 tx=0x7f8db801c970 comp rx=0 tx=0).stop 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 -- 172.21.15.96:0/758374314 shutdown_connections 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 --2- 172.21.15.96:0/758374314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc009e820 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 -- 172.21.15.96:0/758374314 >> 172.21.15.96:0/758374314 conn(0x7f8dc0098a60 msgr2=0x7f8dc009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 -- 172.21.15.96:0/758374314 shutdown_connections 2024-09-13T17:44:37.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.715+0000 7f8dc8341640 1 -- 172.21.15.96:0/758374314 wait complete. 2024-09-13T17:44:37.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 Processor -- start 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 -- start start 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc011c930 con 0x7f8dc0134e70 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc60dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc60dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55604/0 (socket says 172.21.15.96:55604) 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc60dd640 1 -- 172.21.15.96:0/3567747152 learned_addr learned my addr 172.21.15.96:0/3567747152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:37.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc60dd640 1 -- 172.21.15.96:0/3567747152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db8009a70 con 0x7f8dc0134e70 2024-09-13T17:44:37.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc60dd640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8db801a5a0 tx=0x7f8db8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8db80b9790 con 0x7f8dc0134e70 2024-09-13T17:44:37.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8db80b98f0 con 0x7f8dc0134e70 2024-09-13T17:44:37.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc011cb60 con 0x7f8dc0134e70 2024-09-13T17:44:37.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8db80b9bd0 con 0x7f8dc0134e70 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc00a5120 con 0x7f8dc0134e70 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc013b4b0 con 0x7f8dc0134e70 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f8db80b9d30 con 0x7f8dc0134e70 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 0x7f8da4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f8db80e30c0 con 0x7f8dc0134e70 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc58dc640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 0x7f8da4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:37.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.719+0000 7f8dc58dc640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 0x7f8da4037b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8db0001610 tx=0x7f8db0007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:37.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.723+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8db80abb30 con 0x7f8dc0134e70 2024-09-13T17:44:37.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.883+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dc013af00 con 0x7f8dc0134e70 2024-09-13T17:44:37.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.883+0000 7f8db6ffd640 1 -- 172.21.15.96:0/3567747152 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+7979 (secure 0 0 0) 0x7f8db80b2020 con 0x7f8dc0134e70 2024-09-13T17:44:37.893 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:37.894 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":49,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:36.551563+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6826/285199718":"2024-09-14T17:44:33.524153+0000","172.21.15.96:6829/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/2758166216":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/1100369952":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/494405909":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6827/285199718":"2024-09-14T17:44:33.524153+0000","172.21.15.96:6828/2227270685":"2024-09-14T17:44:27.274664+0000","172.21.15.96:0/3234139153":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/252612581":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/2785343960":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3263021005":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2962687900":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/2805890947":"2024-09-14T17:44:21.568899+0000","172.21.15.96:0/715245426":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/1425974344":"2024-09-14T17:44:21.568899+0000","172.21.15.96:6825/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3193678716":"2024-09-14T17:43:51.591620+0000","172.21.15.96:6824/526856805":"2024-09-14T17:43:51.591620+0000","172.21.15.96:0/3614824494":"2024-09-14T17:43:51.591620+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:37.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 msgr2=0x7f8da4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 0x7f8da4037b50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8db0001610 tx=0x7f8db0007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 msgr2=0x7f8dc006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8db801a5a0 tx=0x7f8db8019710 comp rx=0 tx=0).stop 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 shutdown_connections 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f8da4035690 0x7f8da4037b50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 --2- 172.21.15.96:0/3567747152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8dc0134e70 0x7f8dc006f6d0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 >> 172.21.15.96:0/3567747152 conn(0x7f8dc0098a60 msgr2=0x7f8dc0133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 shutdown_connections 2024-09-13T17:44:37.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:37.887+0000 7f8dc8341640 1 -- 172.21.15.96:0/3567747152 wait complete. 2024-09-13T17:44:37.908 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:37.908 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:38.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.011+0000 7f0d7213c640 1 Processor -- start 2024-09-13T17:44:38.018 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.011+0000 7f0d7213c640 1 -- start start 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d7213c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d6c1366a0 0x7f0d6c13ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d7213c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6c136ab0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d7213c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c071cd0 con 0x7f0d6c1366a0 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0d6c1366a0 0x7f0d6c13ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0d6c0a0860 0x7f0d6c135f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54608/0 (socket says 172.21.15.96:54608) 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6b7fe640 1 -- 172.21.15.96:0/1356476772 learned_addr learned my addr 172.21.15.96:0/1356476772 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:38.019 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6affd640 1 -- 172.21.15.96:0/1356476772 >> v1:172.21.15.96:6789/0 conn(0x7f0d6c0a0860 legacy=0x7f0d6c135f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6a7fc640 1 -- 172.21.15.96:0/1356476772 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294321423 0 0) 0x7f0d6c136ab0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.015+0000 7f0d6affd640 1 -- 172.21.15.96:0/1356476772 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0a0c40 con 0x7f0d6c1366a0 2024-09-13T17:44:38.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6affd640 1 --2- 172.21.15.96:0/1356476772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c1366a0 0x7f0d6c13ae90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009740 tx=0x7f0d5401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43ecb923c5595ed server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6a7fc640 1 -- 172.21.15.96:0/1356476772 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d540ab530 con 0x7f0d6c1366a0 2024-09-13T17:44:38.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6a7fc640 1 -- 172.21.15.96:0/1356476772 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d540a7050 con 0x7f0d6c1366a0 2024-09-13T17:44:38.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6a7fc640 1 -- 172.21.15.96:0/1356476772 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d540ab7e0 con 0x7f0d6c1366a0 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/1356476772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c1366a0 msgr2=0x7f0d6c13ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 --2- 172.21.15.96:0/1356476772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c1366a0 0x7f0d6c13ae90 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f0d54009740 tx=0x7f0d5401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/1356476772 shutdown_connections 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 --2- 172.21.15.96:0/1356476772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c1366a0 0x7f0d6c13ae90 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/1356476772 >> 172.21.15.96:0/1356476772 conn(0x7f0d6c0a7390 msgr2=0x7f0d6c0a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/1356476772 shutdown_connections 2024-09-13T17:44:38.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/1356476772 wait complete. 2024-09-13T17:44:38.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 Processor -- start 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- start start 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c13aac0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55620/0 (socket says 172.21.15.96:55620) 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6b7fe640 1 -- 172.21.15.96:0/3819652109 learned_addr learned my addr 172.21.15.96:0/3819652109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6b7fe640 1 -- 172.21.15.96:0/3819652109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d54009420 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d6b7fe640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c840 tx=0x7f0d6400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d6400a660 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400a7e0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6c09c550 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d6400aac0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c09c260 con 0x7f0d6c0a0860 2024-09-13T17:44:38.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f0d6400ac20 con 0x7f0d6c0a0860 2024-09-13T17:44:38.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.019+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6c135f90 con 0x7f0d6c0a0860 2024-09-13T17:44:38.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.023+0000 7f0d63fff640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 0x7f0d4c037b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.023+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f0d6404a0d0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.023+0000 7f0d6affd640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 0x7f0d4c037b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.023+0000 7f0d6affd640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 0x7f0d4c037b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d540ad000 tx=0x7f0d54008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.027+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0d6401a6d0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d6c06b2c0 con 0x7f0d6c0a0860 2024-09-13T17:44:38.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d63fff640 1 -- 172.21.15.96:0/3819652109 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d64018030 con 0x7f0d6c0a0860 2024-09-13T17:44:38.189 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 msgr2=0x7f0d4c037b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d7213c640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 0x7f0d4c037b80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d540ad000 tx=0x7f0d54008040 comp rx=0 tx=0).stop 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 msgr2=0x7f0d6c09bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.183+0000 7f0d7213c640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c840 tx=0x7f0d6400cd00 comp rx=0 tx=0).stop 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 shutdown_connections 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0d4c0356c0 0x7f0d4c037b80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 --2- 172.21.15.96:0/3819652109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d6c0a0860 0x7f0d6c09bd20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 >> 172.21.15.96:0/3819652109 conn(0x7f0d6c0a7390 msgr2=0x7f0d6c132c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 shutdown_connections 2024-09-13T17:44:38.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.187+0000 7f0d7213c640 1 -- 172.21.15.96:0/3819652109 wait complete. 2024-09-13T17:44:38.207 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop' 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7d190640 1 Processor -- start 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7d190640 1 -- start start 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7d190640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcb78134e70 0x7fcb7809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7d190640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb7809ed60 con 0x7fcb78135250 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7d190640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb7809ddc0 con 0x7fcb78134e70 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcb78135250 0x7fcb78135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54616/0 (socket says 172.21.15.96:54616) 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb7659c640 1 -- 172.21.15.96:0/1658727016 learned_addr learned my addr 172.21.15.96:0/1658727016 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb76d9d640 1 --2- 172.21.15.96:0/1658727016 >> v2:172.21.15.96:3300/0 conn(0x7fcb78134e70 0x7fcb7809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb75d9b640 1 -- 172.21.15.96:0/1658727016 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259838835 0 0) 0x7fcb70001120 con 0x7fcb78135250 2024-09-13T17:44:38.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb75d9b640 1 -- 172.21.15.96:0/1658727016 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb64003530 con 0x7fcb78135250 2024-09-13T17:44:38.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb76d9d640 1 -- 172.21.15.96:0/1658727016 >> v1:172.21.15.96:6789/0 conn(0x7fcb78135250 legacy=0x7fcb78135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.315+0000 7fcb76d9d640 1 -- 172.21.15.96:0/1658727016 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780a68b0 con 0x7fcb78134e70 2024-09-13T17:44:38.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 --2- 172.21.15.96:0/1658727016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7809e820 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009d90 tx=0x7fcb6001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d549cce86c8c8a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb75d9b640 1 -- 172.21.15.96:0/1658727016 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb600a57b0 con 0x7fcb78134e70 2024-09-13T17:44:38.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb75d9b640 1 -- 172.21.15.96:0/1658727016 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb600a5910 con 0x7fcb78134e70 2024-09-13T17:44:38.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb75d9b640 1 -- 172.21.15.96:0/1658727016 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb600a5bd0 con 0x7fcb78134e70 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/1658727016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 msgr2=0x7fcb7809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 --2- 172.21.15.96:0/1658727016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7809e820 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fcb60009d90 tx=0x7fcb6001c970 comp rx=0 tx=0).stop 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/1658727016 shutdown_connections 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 --2- 172.21.15.96:0/1658727016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7809e820 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/1658727016 >> 172.21.15.96:0/1658727016 conn(0x7fcb78098a60 msgr2=0x7fcb7809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/1658727016 shutdown_connections 2024-09-13T17:44:38.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/1658727016 wait complete. 2024-09-13T17:44:38.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 Processor -- start 2024-09-13T17:44:38.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- start start 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb780a4f90 con 0x7fcb78134e70 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55636/0 (socket says 172.21.15.96:55636) 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 -- 172.21.15.96:0/2589608351 learned_addr learned my addr 172.21.15.96:0/2589608351 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 -- 172.21.15.96:0/2589608351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb60009a70 con 0x7fcb78134e70 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb76d9d640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcb6001a5a0 tx=0x7fcb60019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb600b9580 con 0x7fcb78134e70 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb600b96e0 con 0x7fcb78134e70 2024-09-13T17:44:38.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb600b99c0 con 0x7fcb78134e70 2024-09-13T17:44:38.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb78070800 con 0x7fcb78134e70 2024-09-13T17:44:38.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.319+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb780711e0 con 0x7fcb78134e70 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fcb600b9b20 con 0x7fcb78134e70 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb781368b0 con 0x7fcb78134e70 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb6f7fe640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 0x7fcb58037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7fcb600e2fc0 con 0x7fcb78134e70 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb7659c640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 0x7fcb58037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.323+0000 7fcb7659c640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 0x7fcb58037b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb70001630 tx=0x7fcb70001160 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.327+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcb60114db0 con 0x7fcb78134e70 2024-09-13T17:44:38.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.535+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop"]} v 0) v1 -- 0x7fcb78071530 con 0x7fcb78134e70 2024-09-13T17:44:38.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.563+0000 7fcb6f7fe640 1 -- 172.21.15.96:0/2589608351 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop"]}]=0 v163) v1 ==== 126+0+0 (secure 0 0 0) 0x7fcb600bd070 con 0x7fcb78134e70 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 msgr2=0x7fcb58037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 0x7fcb58037b50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fcb70001630 tx=0x7fcb70001160 comp rx=0 tx=0).stop 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 msgr2=0x7fcb7806f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fcb6001a5a0 tx=0x7fcb60019710 comp rx=0 tx=0).stop 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 shutdown_connections 2024-09-13T17:44:38.572 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fcb58035690 0x7fcb58037b50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 --2- 172.21.15.96:0/2589608351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb78134e70 0x7fcb7806f6d0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 >> 172.21.15.96:0/2589608351 conn(0x7fcb78098a60 msgr2=0x7fcb78098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 shutdown_connections 2024-09-13T17:44:38.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.567+0000 7fcb7d190640 1 -- 172.21.15.96:0/2589608351 wait complete. 2024-09-13T17:44:38.587 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:44:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee8b9f640 1 Processor -- start 2024-09-13T17:44:38.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee8b9f640 1 -- start start 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee8b9f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7ee4134670 0x7f7ee4134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee8b9f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee4071670 con 0x7f7ee406e5c0 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee8b9f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee4071360 con 0x7f7ee4134670 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7ee4134670 0x7f7ee4134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7ee4134670 0x7f7ee4134a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55642/0 (socket says 172.21.15.96:55642) 2024-09-13T17:44:38.715 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee1d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7ee406e5c0 0x7f7ee406ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54620/0 (socket says 172.21.15.96:54620) 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.703+0000 7f7ee259c640 1 -- 172.21.15.96:0/3857661736 learned_addr learned my addr 172.21.15.96:0/3857661736 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee259c640 1 -- 172.21.15.96:0/3857661736 >> v1:172.21.15.96:6789/0 conn(0x7f7ee406e5c0 legacy=0x7f7ee406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee159a640 1 -- 172.21.15.96:0/3857661736 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894287516 0 0) 0x7f7ee4071670 con 0x7f7ee406e5c0 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee259c640 1 -- 172.21.15.96:0/3857661736 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee40a7200 con 0x7f7ee4134670 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee259c640 1 --2- 172.21.15.96:0/3857661736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee4134670 0x7f7ee4134a50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a990 tx=0x7f7ed401d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3883f9e9df09b8e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee159a640 1 -- 172.21.15.96:0/3857661736 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed40a66b0 con 0x7f7ee4134670 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee159a640 1 -- 172.21.15.96:0/3857661736 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed40a6810 con 0x7f7ee4134670 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee159a640 1 -- 172.21.15.96:0/3857661736 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed40a6ad0 con 0x7f7ee4134670 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/3857661736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee4134670 msgr2=0x7f7ee4134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/3857661736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee4134670 0x7f7ee4134a50 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400a990 tx=0x7f7ed401d820 comp rx=0 tx=0).stop 2024-09-13T17:44:38.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/3857661736 shutdown_connections 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/3857661736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee4134670 0x7f7ee4134a50 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/3857661736 >> 172.21.15.96:0/3857661736 conn(0x7f7ee4098a60 msgr2=0x7f7ee409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/3857661736 shutdown_connections 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.707+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/3857661736 wait complete. 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 Processor -- start 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 -- start start 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee40a6970 con 0x7f7ee406e5c0 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55658/0 (socket says 172.21.15.96:55658) 2024-09-13T17:44:38.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee259c640 1 -- 172.21.15.96:0/4042567840 learned_addr learned my addr 172.21.15.96:0/4042567840 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee259c640 1 -- 172.21.15.96:0/4042567840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed400a670 con 0x7f7ee406e5c0 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee259c640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7ed401a430 tx=0x7f7ed401c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed4004150 con 0x7f7ee406e5c0 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed40042b0 con 0x7f7ee406e5c0 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed40ae790 con 0x7f7ee406e5c0 2024-09-13T17:44:38.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee40a1fc0 con 0x7f7ee406e5c0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee40a3250 con 0x7f7ee406e5c0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee4070290 con 0x7f7ee406e5c0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f7ed40b5040 con 0x7f7ee406e5c0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 0x7f7ec4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f7ed40e2790 con 0x7f7ee406e5c0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee1d9b640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 0x7f7ec4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:38.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.711+0000 7f7ee1d9b640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 0x7f7ec4037b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007fd0 tx=0x7f7ecc007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:38.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.715+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7ee4070290 con 0x7f7ee406e5c0 2024-09-13T17:44:38.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.863+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7ee4070290 con 0x7f7ee406e5c0 2024-09-13T17:44:38.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.863+0000 7f7edaffd640 1 -- 172.21.15.96:0/4042567840 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v9)=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7ee4070290 con 0x7f7ee406e5c0 2024-09-13T17:44:38.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 msgr2=0x7f7ec4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 0x7f7ec4037b50 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc007fd0 tx=0x7f7ecc007920 comp rx=0 tx=0).stop 2024-09-13T17:44:38.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 msgr2=0x7f7ee40a2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f7ed401a430 tx=0x7f7ed401c060 comp rx=0 tx=0).stop 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 shutdown_connections 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7ec4035690 0x7f7ec4037b50 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 --2- 172.21.15.96:0/4042567840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ee406e5c0 0x7f7ee40a2d10 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 >> 172.21.15.96:0/4042567840 conn(0x7f7ee4098a60 msgr2=0x7f7ee4139bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 shutdown_connections 2024-09-13T17:44:38.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:38.867+0000 7f7ee8b9f640 1 -- 172.21.15.96:0/4042567840 wait complete. 2024-09-13T17:44:38.887 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:44:38.888 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:44:38.964 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:44:38.964 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:38.964 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:38.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:44:38.969 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:44:38.969 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:44:39.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:44:39.015 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-09-13T17:44:39.015 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:39.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f933ba0a640 1 Processor -- start 2024-09-13T17:44:39.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f933ba0a640 1 -- start start 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f933ba0a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f933409e820 0x7f933413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f933ba0a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340a68b0 con 0x7f9334134e70 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f933ba0a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f933409ddc0 con 0x7f933409e820 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f9338fa5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f933409e820 0x7f933413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f93397a6640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9334134e70 0x7f9334135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:54628/0 (socket says 172.21.15.96:54628) 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.175+0000 7f93397a6640 1 -- 172.21.15.96:0/3878946215 learned_addr learned my addr 172.21.15.96:0/3878946215 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f9338fa5640 1 -- 172.21.15.96:0/3878946215 >> v1:172.21.15.96:6789/0 conn(0x7f9334134e70 legacy=0x7f9334135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:39.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f9338fa5640 1 -- 172.21.15.96:0/3878946215 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334136970 con 0x7f933409e820 2024-09-13T17:44:39.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f9338fa5640 1 --2- 172.21.15.96:0/3878946215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933413ae50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9324009740 tx=0x7f932401c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05a8d223068ea18 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:39.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f932bfff640 1 -- 172.21.15.96:0/3878946215 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93240ab530 con 0x7f933409e820 2024-09-13T17:44:39.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f932bfff640 1 -- 172.21.15.96:0/3878946215 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93240a7050 con 0x7f933409e820 2024-09-13T17:44:39.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f932bfff640 1 -- 172.21.15.96:0/3878946215 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93240ab7e0 con 0x7f933409e820 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 -- 172.21.15.96:0/3878946215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 msgr2=0x7f933413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3878946215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933413ae50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9324009740 tx=0x7f932401c2b0 comp rx=0 tx=0).stop 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 -- 172.21.15.96:0/3878946215 shutdown_connections 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3878946215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933413ae50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 -- 172.21.15.96:0/3878946215 >> 172.21.15.96:0/3878946215 conn(0x7f9334098a60 msgr2=0x7f933409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 -- 172.21.15.96:0/3878946215 shutdown_connections 2024-09-13T17:44:39.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.179+0000 7f933ba0a640 1 -- 172.21.15.96:0/3878946215 wait complete. 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 Processor -- start 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 -- start start 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933411d370 con 0x7f933409e820 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f93397a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f93397a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55676/0 (socket says 172.21.15.96:55676) 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f93397a6640 1 -- 172.21.15.96:0/3162372561 learned_addr learned my addr 172.21.15.96:0/3162372561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f93397a6640 1 -- 172.21.15.96:0/3162372561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324009420 con 0x7f933409e820 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f93397a6640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f933000efd0 tx=0x7f933000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93300078f0 con 0x7f933409e820 2024-09-13T17:44:39.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330007a50 con 0x7f933409e820 2024-09-13T17:44:39.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933411d5a0 con 0x7f933409e820 2024-09-13T17:44:39.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9330007d30 con 0x7f933409e820 2024-09-13T17:44:39.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93340a5b60 con 0x7f933409e820 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f9330009070 con 0x7f933409e820 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933411e010 con 0x7f933409e820 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 0x7f9318037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f9330049b00 con 0x7f933409e820 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9338fa5640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 0x7f9318037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:39.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.183+0000 7f9338fa5640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 0x7f9318037b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93240ad000 tx=0x7f9324008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:39.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.187+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f93300126f0 con 0x7f933409e820 2024-09-13T17:44:39.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.379+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93340a5630 con 0x7f933409e820 2024-09-13T17:44:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.379+0000 7f9329ffb640 1 -- 172.21.15.96:0/3162372561 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2015 (secure 0 0 0) 0x7f9330019020 con 0x7f933409e820 2024-09-13T17:44:39.386 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:39.386 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5086,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/1651635905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1651635905},{"type":"v1","addr":"172.21.15.96:6829","nonce":1651635905}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5127,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3254739775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3254739775},{"type":"v1","addr":"172.21.15.96:6827","nonce":3254739775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2024-09-13T17:44:39.386 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 29 2024-09-13T17:44:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 msgr2=0x7f9318037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 0x7f9318037b50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f93240ad000 tx=0x7f9324008040 comp rx=0 tx=0).stop 2024-09-13T17:44:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 msgr2=0x7f933411ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:39.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f933000efd0 tx=0x7f933000c430 comp rx=0 tx=0).stop 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 shutdown_connections 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f9318035690 0x7f9318037b50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 --2- 172.21.15.96:0/3162372561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f933409e820 0x7f933411ea70 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 >> 172.21.15.96:0/3162372561 conn(0x7f9334098a60 msgr2=0x7f933406cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 shutdown_connections 2024-09-13T17:44:39.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:39.383+0000 7f933ba0a640 1 -- 172.21.15.96:0/3162372561 wait complete. 2024-09-13T17:44:39.405 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:44:39.405 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:44:39.405 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:44:39.405 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:39.406 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:44:39.406 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T17:44:39.406 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:44:39.406 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:44:39.419 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:44:39.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:44:39.464 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:44:39.464 DEBUG:teuthology.parallel:result is None 2024-09-13T17:44:39.524 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:44:44.644 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-13T17:44:45.419 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:44:45.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:44:45.422 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:44:45.423 DEBUG:teuthology.parallel:result is None 2024-09-13T17:44:45.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:44:45.498 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:44:45.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f057a47e640 1 Processor -- start 2024-09-13T17:44:45.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f057a47e640 1 -- start start 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f057a47e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f057406f6d0 0x7f0574134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f057a47e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f057406e130 con 0x7f057409dc20 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f057a47e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05740a6750 con 0x7f057406f6d0 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0573fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f057406f6d0 0x7f0574134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f05737fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f057409dc20 0x7f0574134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43532/0 (socket says 172.21.15.96:43532) 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f05737fe640 1 -- 172.21.15.96:0/229917917 learned_addr learned my addr 172.21.15.96:0/229917917 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:45.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0573fff640 1 -- 172.21.15.96:0/229917917 >> v1:172.21.15.96:6789/0 conn(0x7f057409dc20 legacy=0x7f0574134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:45.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0573fff640 1 -- 172.21.15.96:0/229917917 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740712e0 con 0x7f057406f6d0 2024-09-13T17:44:45.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0573fff640 1 --2- 172.21.15.96:0/229917917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f0574134ab0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0564009740 tx=0x7f056401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e046028aa7cdf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:45.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0572ffd640 1 -- 172.21.15.96:0/229917917 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05640ab400 con 0x7f057406f6d0 2024-09-13T17:44:45.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0572ffd640 1 -- 172.21.15.96:0/229917917 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05640a7050 con 0x7f057406f6d0 2024-09-13T17:44:45.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.539+0000 7f0572ffd640 1 -- 172.21.15.96:0/229917917 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05640ab6b0 con 0x7f057406f6d0 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/229917917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 msgr2=0x7f0574134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 --2- 172.21.15.96:0/229917917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f0574134ab0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f0564009740 tx=0x7f056401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/229917917 shutdown_connections 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 --2- 172.21.15.96:0/229917917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f0574134ab0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/229917917 >> 172.21.15.96:0/229917917 conn(0x7f0574098a60 msgr2=0x7f057409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:45.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/229917917 shutdown_connections 2024-09-13T17:44:45.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/229917917 wait complete. 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 Processor -- start 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- start start 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740a6180 con 0x7f057406f6d0 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0573fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0573fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34976/0 (socket says 172.21.15.96:34976) 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0573fff640 1 -- 172.21.15.96:0/2618993706 learned_addr learned my addr 172.21.15.96:0/2618993706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0573fff640 1 -- 172.21.15.96:0/2618993706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009420 con 0x7f057406f6d0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0573fff640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0564019fc0 tx=0x7f056401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05640b7450 con 0x7f057406f6d0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05640b75b0 con 0x7f057406f6d0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05740a36a0 con 0x7f057406f6d0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f05640b7850 con 0x7f057406f6d0 2024-09-13T17:44:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05740a5200 con 0x7f057406f6d0 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0574134ab0 con 0x7f057406f6d0 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f05640b79b0 con 0x7f057406f6d0 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 0x7f0560037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 3447+0+0 (secure 0 0 0) 0x7f05640e2420 con 0x7f057406f6d0 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.543+0000 7f05737fe640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 0x7f0560037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:45.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.547+0000 7f05737fe640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 0x7f0560037b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0568001610 tx=0x7f0568007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:45.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.551+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f05640c76c0 con 0x7f057406f6d0 2024-09-13T17:44:45.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:45.723+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0574131840 con 0x7f057406f6d0 2024-09-13T17:44:46.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.527+0000 7f0570ff9640 1 -- 172.21.15.96:0/2618993706 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) v1 ==== 97+0+0 (secure 0 0 0) 0x7f05640b0020 con 0x7f057406f6d0 2024-09-13T17:44:46.532 INFO:teuthology.orchestra.run.smithi096.stderr: removed all blocklist entries 2024-09-13T17:44:46.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 msgr2=0x7f0560037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 0x7f0560037b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0568001610 tx=0x7f0568007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 msgr2=0x7f05740a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f0564019fc0 tx=0x7f056401abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 shutdown_connections 2024-09-13T17:44:46.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f0560035690 0x7f0560037b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 --2- 172.21.15.96:0/2618993706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f057406f6d0 0x7f05740a16b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 >> 172.21.15.96:0/2618993706 conn(0x7f0574098a60 msgr2=0x7f0574134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:46.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 shutdown_connections 2024-09-13T17:44:46.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.531+0000 7f057a47e640 1 -- 172.21.15.96:0/2618993706 wait complete. 2024-09-13T17:44:46.557 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:44:46.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.675+0000 7f49555b6640 1 Processor -- start 2024-09-13T17:44:46.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.675+0000 7f49555b6640 1 -- start start 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f49555b6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4950134670 0x7f4950134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f49555b6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4950071670 con 0x7f495006e5c0 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f49555b6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4950071360 con 0x7f4950134670 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f494effd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4950134670 0x7f4950134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f49467fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f495006e5c0 0x7f495006ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43546/0 (socket says 172.21.15.96:43546) 2024-09-13T17:44:46.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f49467fc640 1 -- 172.21.15.96:0/3606956542 learned_addr learned my addr 172.21.15.96:0/3606956542 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f494effd640 1 -- 172.21.15.96:0/3606956542 >> v1:172.21.15.96:6789/0 conn(0x7f495006e5c0 legacy=0x7f495006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f494e7fc640 1 -- 172.21.15.96:0/3606956542 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264941736 0 0) 0x7f4950071670 con 0x7f495006e5c0 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.679+0000 7f494effd640 1 -- 172.21.15.96:0/3606956542 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500a7200 con 0x7f4950134670 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 --2- 172.21.15.96:0/3606956542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4950134670 0x7f4950134a50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4940009740 tx=0x7f494001e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c03257f951a69ec server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494e7fc640 1 -- 172.21.15.96:0/3606956542 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49400ad580 con 0x7f4950134670 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494e7fc640 1 -- 172.21.15.96:0/3606956542 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400ad6e0 con 0x7f4950134670 2024-09-13T17:44:46.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494e7fc640 1 -- 172.21.15.96:0/3606956542 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49400ad9a0 con 0x7f4950134670 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/3606956542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4950134670 msgr2=0x7f4950134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 --2- 172.21.15.96:0/3606956542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4950134670 0x7f4950134a50 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4940009740 tx=0x7f494001e5b0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/3606956542 shutdown_connections 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 --2- 172.21.15.96:0/3606956542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4950134670 0x7f4950134a50 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/3606956542 >> 172.21.15.96:0/3606956542 conn(0x7f4950098a60 msgr2=0x7f495009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/3606956542 shutdown_connections 2024-09-13T17:44:46.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/3606956542 wait complete. 2024-09-13T17:44:46.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 Processor -- start 2024-09-13T17:44:46.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- start start 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49501cc490 con 0x7f495006e5c0 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:34986/0 (socket says 172.21.15.96:34986) 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 -- 172.21.15.96:0/1610042273 learned_addr learned my addr 172.21.15.96:0/1610042273 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:46.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 -- 172.21.15.96:0/1610042273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4940009420 con 0x7f495006e5c0 2024-09-13T17:44:46.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f494effd640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f494001b1c0 tx=0x7f494001cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:46.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4940004150 con 0x7f495006e5c0 2024-09-13T17:44:46.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49400042b0 con 0x7f495006e5c0 2024-09-13T17:44:46.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500a5620 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4940003620 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.683+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49500a62c0 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f4940003780 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49500a5310 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f4947fff640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 0x7f4930037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f49400e2980 con 0x7f495006e5c0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f49467fc640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 0x7f4930037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:46.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.687+0000 7f49467fc640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 0x7f4930037b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4938007fd0 tx=0x7f4938007920 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:46.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.691+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f49400b6600 con 0x7f495006e5c0 2024-09-13T17:44:46.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.891+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f49500a6510 con 0x7f495006e5c0 2024-09-13T17:44:46.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.891+0000 7f4947fff640 1 -- 172.21.15.96:0/1610042273 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+2534 (secure 0 0 0) 0x7f49400bf3b0 con 0x7f495006e5c0 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_dump": [ 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.a", 2024-09-13T17:44:46.899 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmAVcqLxAAIRo/psndnU3y/VG+/goNRQ==", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.b", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmpF6ANxAAzrn1jVVSCWKv5q1kdChbWA==", 2024-09-13T17:44:46.900 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.901 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.0", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBKeeRmkXcoFhAAdX1HBKKYCHqtacYyAZbF2w==", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.1", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBNeeRmh189GxAAFDpo1k0lxmmillsXoiKXMA==", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:44:46.902 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.2", 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBQeeRmKpvLIhAA4Y7okMAZAQ9cjYpkaKF/kA==", 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:44:46.903 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.0", 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRm3enLAxAAZGIwl+Bs06GY4JG0dbxLBQ==", 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow r", 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T17:44:46.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.1", 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRmNlIQCRAAFFcBZ0Pwtq8kXtBAAFkuDA==", 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-13T17:44:46.905 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.admin", 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmUwoTGBAAycvCNfOc1ueY+QjBkpAqpA==", 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.x", 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmOOcEKhAAJmefIyHpVdx4zDzhwQcawA==", 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:44:46.907 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:46.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 msgr2=0x7f4930037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 0x7f4930037b50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4938007fd0 tx=0x7f4938007920 comp rx=0 tx=0).stop 2024-09-13T17:44:46.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 msgr2=0x7f49501cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:46.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f494001b1c0 tx=0x7f494001cdf0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 shutdown_connections 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f4930035690 0x7f4930037b50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 --2- 172.21.15.96:0/1610042273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f495006e5c0 0x7f49501cfa00 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 >> 172.21.15.96:0/1610042273 conn(0x7f4950098a60 msgr2=0x7f495009a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 shutdown_connections 2024-09-13T17:44:46.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:46.895+0000 7f49555b6640 1 -- 172.21.15.96:0/1610042273 wait complete. 2024-09-13T17:44:46.915 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:47.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.015+0000 7f650fd24640 1 Processor -- start 2024-09-13T17:44:47.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.015+0000 7f650fd24640 1 -- start start 2024-09-13T17:44:47.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650fd24640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f650806f6d0 0x7f6508134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650fd24640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f650806e130 con 0x7f650809dc20 2024-09-13T17:44:47.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650fd24640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65080a6750 con 0x7f650806f6d0 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650dac0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f650806f6d0 0x7f6508134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650d2bf640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f650809dc20 0x7f6508134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43560/0 (socket says 172.21.15.96:43560) 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650d2bf640 1 -- 172.21.15.96:0/192786451 learned_addr learned my addr 172.21.15.96:0/192786451 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650dac0640 1 -- 172.21.15.96:0/192786451 >> v1:172.21.15.96:6789/0 conn(0x7f650809dc20 legacy=0x7f6508134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650cabe640 1 -- 172.21.15.96:0/192786451 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533975803 0 0) 0x7f650806e130 con 0x7f650809dc20 2024-09-13T17:44:47.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650dac0640 1 -- 172.21.15.96:0/192786451 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65080712e0 con 0x7f650806f6d0 2024-09-13T17:44:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650dac0640 1 --2- 172.21.15.96:0/192786451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f6508134ab0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009740 tx=0x7f64f801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31b07983ab6528e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650cabe640 1 -- 172.21.15.96:0/192786451 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f80ab400 con 0x7f650806f6d0 2024-09-13T17:44:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650cabe640 1 -- 172.21.15.96:0/192786451 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f80a7050 con 0x7f650806f6d0 2024-09-13T17:44:47.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.019+0000 7f650cabe640 1 -- 172.21.15.96:0/192786451 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f80ab6b0 con 0x7f650806f6d0 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/192786451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 msgr2=0x7f6508134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 --2- 172.21.15.96:0/192786451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f6508134ab0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009740 tx=0x7f64f801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/192786451 shutdown_connections 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 --2- 172.21.15.96:0/192786451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f6508134ab0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/192786451 >> 172.21.15.96:0/192786451 conn(0x7f6508098a60 msgr2=0x7f650809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/192786451 shutdown_connections 2024-09-13T17:44:47.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/192786451 wait complete. 2024-09-13T17:44:47.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 Processor -- start 2024-09-13T17:44:47.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- start start 2024-09-13T17:44:47.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65080a0640 con 0x7f650806f6d0 2024-09-13T17:44:47.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650dac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650dac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35016/0 (socket says 172.21.15.96:35016) 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650dac0640 1 -- 172.21.15.96:0/1861092584 learned_addr learned my addr 172.21.15.96:0/1861092584 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650dac0640 1 -- 172.21.15.96:0/1861092584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f8009420 con 0x7f650806f6d0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650dac0640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f64f8019fc0 tx=0x7f64f801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f80b7450 con 0x7f650806f6d0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f80b75b0 con 0x7f650806f6d0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650812f930 con 0x7f650806f6d0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64f80b7850 con 0x7f650806f6d0 2024-09-13T17:44:47.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f650806f480 con 0x7f650806f6d0 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f64f80b79b0 con 0x7f650806f6d0 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.023+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650806eba0 con 0x7f650806f6d0 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.027+0000 7f64fe7fc640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 0x7f64ec037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.027+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f64f80e6ba0 con 0x7f650806f6d0 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.027+0000 7f650d2bf640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 0x7f64ec037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.027+0000 7f650d2bf640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 0x7f64ec037b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6504007fd0 tx=0x7f6504007940 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.031+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f64f80c76c0 con 0x7f650806f6d0 2024-09-13T17:44:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.183+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6508134ab0 con 0x7f650806f6d0 2024-09-13T17:44:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f64fe7fc640 1 -- 172.21.15.96:0/1861092584 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+6723 (secure 0 0 0) 0x7f64f80bb020 con 0x7f650806f6d0 2024-09-13T17:44:47.191 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:47.191 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":52,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:46.523457+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 msgr2=0x7f64ec037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 0x7f64ec037b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6504007fd0 tx=0x7f6504007940 comp rx=0 tx=0).stop 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 msgr2=0x7f65080a4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f64f8019fc0 tx=0x7f64f801abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 shutdown_connections 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f64ec035690 0x7f64ec037b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 --2- 172.21.15.96:0/1861092584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f650806f6d0 0x7f65080a4b90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 >> 172.21.15.96:0/1861092584 conn(0x7f6508098a60 msgr2=0x7f6508098f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.187+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 shutdown_connections 2024-09-13T17:44:47.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.191+0000 7f650fd24640 1 -- 172.21.15.96:0/1861092584 wait complete. 2024-09-13T17:44:47.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:44:47.211 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:44:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e7d82f640 1 Processor -- start 2024-09-13T17:44:47.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e7d82f640 1 -- start start 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e7d82f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3e7809d690 0x7f3e78138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e7d82f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e78138050 con 0x7f3e781387d0 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e7d82f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7809c410 con 0x7f3e7809d690 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e767fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3e781387d0 0x7f3e78134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43568/0 (socket says 172.21.15.96:43568) 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e76ffd640 1 --2- 172.21.15.96:0/4073397373 >> v2:172.21.15.96:3300/0 conn(0x7f3e7809d690 0x7f3e78138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e767fc640 1 -- 172.21.15.96:0/4073397373 learned_addr learned my addr 172.21.15.96:0/4073397373 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e75ffb640 1 -- 172.21.15.96:0/4073397373 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212904209 0 0) 0x7f3e6c001120 con 0x7f3e781387d0 2024-09-13T17:44:47.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e75ffb640 1 -- 172.21.15.96:0/4073397373 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e60003530 con 0x7f3e781387d0 2024-09-13T17:44:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e76ffd640 1 -- 172.21.15.96:0/4073397373 >> v1:172.21.15.96:6789/0 conn(0x7f3e781387d0 legacy=0x7f3e78134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.319+0000 7f3e76ffd640 1 -- 172.21.15.96:0/4073397373 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7809cba0 con 0x7f3e7809d690 2024-09-13T17:44:47.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 --2- 172.21.15.96:0/4073397373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78138290 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009740 tx=0x7f3e6801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235c50cd76092b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e75ffb640 1 -- 172.21.15.96:0/4073397373 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e680a8840 con 0x7f3e7809d690 2024-09-13T17:44:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e75ffb640 1 -- 172.21.15.96:0/4073397373 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e680a89a0 con 0x7f3e7809d690 2024-09-13T17:44:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e75ffb640 1 -- 172.21.15.96:0/4073397373 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e680a8c60 con 0x7f3e7809d690 2024-09-13T17:44:47.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/4073397373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 msgr2=0x7f3e78138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/4073397373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78138290 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f3e68009740 tx=0x7f3e6801ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/4073397373 shutdown_connections 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/4073397373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78138290 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/4073397373 >> 172.21.15.96:0/4073397373 conn(0x7f3e7806bc60 msgr2=0x7f3e7806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/4073397373 shutdown_connections 2024-09-13T17:44:47.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/4073397373 wait complete. 2024-09-13T17:44:47.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 Processor -- start 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- start start 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e780a6fa0 con 0x7f3e7809d690 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35042/0 (socket says 172.21.15.96:35042) 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 -- 172.21.15.96:0/3074435630 learned_addr learned my addr 172.21.15.96:0/3074435630 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 -- 172.21.15.96:0/3074435630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68009420 con 0x7f3e7809d690 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e76ffd640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3e6801c840 tx=0x7f3e6801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e680b0780 con 0x7f3e7809d690 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e680b08e0 con 0x7f3e7809d690 2024-09-13T17:44:47.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e780a2a90 con 0x7f3e7809d690 2024-09-13T17:44:47.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3e680b0b80 con 0x7f3e7809d690 2024-09-13T17:44:47.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.323+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e780a2780 con 0x7f3e7809d690 2024-09-13T17:44:47.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f3e680b7040 con 0x7f3e7809d690 2024-09-13T17:44:47.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e5ffff640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 0x7f3e64037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7809bc00 con 0x7f3e7809d690 2024-09-13T17:44:47.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f3e680e53e0 con 0x7f3e7809d690 2024-09-13T17:44:47.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e767fc640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 0x7f3e64037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.327+0000 7f3e767fc640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 0x7f3e64037b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c005890 tx=0x7f3e6c001190 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.331+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3e7809bc00 con 0x7f3e7809d690 2024-09-13T17:44:47.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.475+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e7809bbd0 con 0x7f3e7809d690 2024-09-13T17:44:47.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.475+0000 7f3e5ffff640 1 -- 172.21.15.96:0/3074435630 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e680aed10 con 0x7f3e7809d690 2024-09-13T17:44:47.482 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:44:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 msgr2=0x7f3e64037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 0x7f3e64037b50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c005890 tx=0x7f3e6c001190 comp rx=0 tx=0).stop 2024-09-13T17:44:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 msgr2=0x7f3e78131450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f3e6801c840 tx=0x7f3e6801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 shutdown_connections 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3e64035690 0x7f3e64037b50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 --2- 172.21.15.96:0/3074435630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7809d690 0x7f3e78131450 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 >> 172.21.15.96:0/3074435630 conn(0x7f3e7806bc60 msgr2=0x7f3e781bfde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 shutdown_connections 2024-09-13T17:44:47.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.479+0000 7f3e7d82f640 1 -- 172.21.15.96:0/3074435630 wait complete. 2024-09-13T17:44:47.498 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:44:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7964396640 1 Processor -- start 2024-09-13T17:44:47.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7964396640 1 -- start start 2024-09-13T17:44:47.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7964396640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f795c09e820 0x7f795c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7964396640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c0a68b0 con 0x7f795c134e70 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7964396640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c09ddc0 con 0x7f795c09e820 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7962132640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f795c134e70 0x7f795c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43574/0 (socket says 172.21.15.96:43574) 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7962132640 1 -- 172.21.15.96:0/2075455124 learned_addr learned my addr 172.21.15.96:0/2075455124 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7961931640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f795c09e820 0x7f795c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7961931640 1 -- 172.21.15.96:0/2075455124 >> v1:172.21.15.96:6789/0 conn(0x7f795c134e70 legacy=0x7f795c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7961931640 1 -- 172.21.15.96:0/2075455124 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c136970 con 0x7f795c09e820 2024-09-13T17:44:47.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.607+0000 7f7961130640 1 -- 172.21.15.96:0/2075455124 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024632087 0 0) 0x7f795c0a68b0 con 0x7f795c134e70 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7961931640 1 --2- 172.21.15.96:0/2075455124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c13ae50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7950009740 tx=0x7f795001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c55c9b9d7113f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7961130640 1 -- 172.21.15.96:0/2075455124 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79500ab530 con 0x7f795c09e820 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7961130640 1 -- 172.21.15.96:0/2075455124 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79500a7050 con 0x7f795c09e820 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7961130640 1 -- 172.21.15.96:0/2075455124 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79500ab7e0 con 0x7f795c09e820 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- 172.21.15.96:0/2075455124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 msgr2=0x7f795c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 --2- 172.21.15.96:0/2075455124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c13ae50 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f7950009740 tx=0x7f795001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- 172.21.15.96:0/2075455124 shutdown_connections 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 --2- 172.21.15.96:0/2075455124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c13ae50 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- 172.21.15.96:0/2075455124 >> 172.21.15.96:0/2075455124 conn(0x7f795c098a60 msgr2=0x7f795c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- 172.21.15.96:0/2075455124 shutdown_connections 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- 172.21.15.96:0/2075455124 wait complete. 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 Processor -- start 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- start start 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7964396640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0a5010 con 0x7f795c09e820 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7962132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7962132640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35068/0 (socket says 172.21.15.96:35068) 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7962132640 1 -- 172.21.15.96:0/820586542 learned_addr learned my addr 172.21.15.96:0/820586542 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7962132640 1 -- 172.21.15.96:0/820586542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950009420 con 0x7f795c09e820 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f7962132640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f795400c420 tx=0x7f795400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.611+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f79540074d0 con 0x7f795c09e820 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7954007630 con 0x7f795c09e820 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c070380 con 0x7f795c09e820 2024-09-13T17:44:47.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7954004960 con 0x7f795c09e820 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c11d9b0 con 0x7f795c09e820 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f7954004ac0 con 0x7f795c09e820 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c11e100 con 0x7f795c09e820 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f795affd640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 0x7f7948037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f7954046b70 con 0x7f795c09e820 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f7961931640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 0x7f7948037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.615+0000 7f7961931640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 0x7f7948037b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79500ad000 tx=0x7f7950008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.619+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f795c11e100 con 0x7f795c09e820 2024-09-13T17:44:47.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f795c070380 con 0x7f795c09e820 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f795affd640 1 -- 172.21.15.96:0/820586542 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) v1 ==== 84+0+65 (secure 0 0 0) 0x7f795c070380 con 0x7f795c09e820 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:44:47.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:47.773 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 msgr2=0x7f7948037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 0x7f7948037b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f79500ad000 tx=0x7f7950008040 comp rx=0 tx=0).stop 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 msgr2=0x7f795c0a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f795400c420 tx=0x7f795400c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 shutdown_connections 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7948035690 0x7f7948037b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 --2- 172.21.15.96:0/820586542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f795c09e820 0x7f795c0a3ab0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 >> 172.21.15.96:0/820586542 conn(0x7f795c098a60 msgr2=0x7f795c134100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 shutdown_connections 2024-09-13T17:44:47.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.767+0000 7f7964396640 1 -- 172.21.15.96:0/820586542 wait complete. 2024-09-13T17:44:47.789 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:44:47.789 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:44:47.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ababe4640 1 Processor -- start 2024-09-13T17:44:47.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ababe4640 1 -- start start 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ababe4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5ab409e890 0x7f5ab413ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ababe4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab409e0d0 con 0x7f5ab4135e80 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ababe4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab409ef50 con 0x7f5ab409e890 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5ab409e890 0x7f5ab413ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab8980640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5ab4135e80 0x7f5ab4136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43578/0 (socket says 172.21.15.96:43578) 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab8980640 1 -- 172.21.15.96:0/2156748561 learned_addr learned my addr 172.21.15.96:0/2156748561 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab3fff640 1 -- 172.21.15.96:0/2156748561 >> v1:172.21.15.96:6789/0 conn(0x7f5ab4135e80 legacy=0x7f5ab4136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab37fe640 1 -- 172.21.15.96:0/2156748561 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194513142 0 0) 0x7f5ab409e0d0 con 0x7f5ab4135e80 2024-09-13T17:44:47.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.903+0000 7f5ab3fff640 1 -- 172.21.15.96:0/2156748561 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab409bf40 con 0x7f5ab409e890 2024-09-13T17:44:47.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab3fff640 1 --2- 172.21.15.96:0/2156748561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab413ade0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009740 tx=0x7f5aa801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9447f4cb2f1348f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab37fe640 1 -- 172.21.15.96:0/2156748561 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5aa80ab570 con 0x7f5ab409e890 2024-09-13T17:44:47.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab37fe640 1 -- 172.21.15.96:0/2156748561 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa80a7070 con 0x7f5ab409e890 2024-09-13T17:44:47.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab37fe640 1 -- 172.21.15.96:0/2156748561 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5aa80ab820 con 0x7f5ab409e890 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2156748561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 msgr2=0x7f5ab413ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2156748561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab413ade0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009740 tx=0x7f5aa801c420 comp rx=0 tx=0).stop 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2156748561 shutdown_connections 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2156748561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab413ade0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2156748561 >> 172.21.15.96:0/2156748561 conn(0x7f5ab4098a60 msgr2=0x7f5ab409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2156748561 shutdown_connections 2024-09-13T17:44:47.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2156748561 wait complete. 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 Processor -- start 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- start start 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab40a3c80 con 0x7f5ab409e890 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab8980640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab8980640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35096/0 (socket says 172.21.15.96:35096) 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab8980640 1 -- 172.21.15.96:0/2279045612 learned_addr learned my addr 172.21.15.96:0/2279045612 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab8980640 1 -- 172.21.15.96:0/2279045612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa8009420 con 0x7f5ab409e890 2024-09-13T17:44:47.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab8980640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c390 tx=0x7f5aa400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5aa4007410 con 0x7f5ab409e890 2024-09-13T17:44:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aa4007590 con 0x7f5ab409e890 2024-09-13T17:44:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5aa40047f0 con 0x7f5ab409e890 2024-09-13T17:44:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab40a21c0 con 0x7f5ab409e890 2024-09-13T17:44:47.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab41c9b40 con 0x7f5ab409e890 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ab40a6380 con 0x7f5ab409e890 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f5aa4004950 con 0x7f5ab409e890 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ab17fa640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 0x7f5aa0037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f5aa40474d0 con 0x7f5ab409e890 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ab3fff640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 0x7f5aa0037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:47.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.911+0000 7f5ab3fff640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 0x7f5aa0037b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80af000 tx=0x7f5aa8008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:47.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:47.915+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5ab40a6380 con 0x7f5ab409e890 2024-09-13T17:44:48.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.075+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5ab40a6380 con 0x7f5ab409e890 2024-09-13T17:44:48.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.903+0000 7f5ab17fa640 1 -- 172.21.15.96:0/2279045612 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5ab40a6380 con 0x7f5ab409e890 2024-09-13T17:44:48.909 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-13T17:44:48.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 msgr2=0x7f5aa0037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:48.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.907+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 0x7f5aa0037b50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80af000 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2024-09-13T17:44:48.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.907+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 msgr2=0x7f5ab40a2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:48.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.907+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f5aa400c390 tx=0x7f5aa400c850 comp rx=0 tx=0).stop 2024-09-13T17:44:48.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 shutdown_connections 2024-09-13T17:44:48.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f5aa0035690 0x7f5aa0037b50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 --2- 172.21.15.96:0/2279045612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ab409e890 0x7f5ab40a2ec0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:48.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 >> 172.21.15.96:0/2279045612 conn(0x7f5ab4098a60 msgr2=0x7f5ab409a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:48.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 shutdown_connections 2024-09-13T17:44:48.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:48.911+0000 7f5ababe4640 1 -- 172.21.15.96:0/2279045612 wait complete. 2024-09-13T17:44:48.941 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:44:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e79c9640 1 Processor -- start 2024-09-13T17:44:49.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e79c9640 1 -- start start 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e79c9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66e013c4d0 0x7f66e006ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e79c9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e00710a0 con 0x7f66e013c8b0 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e79c9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e013c160 con 0x7f66e013c4d0 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66e013c4d0 0x7f66e006ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f66e013c4d0 0x7f66e006ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35112/0 (socket says 172.21.15.96:35112) 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 -- 172.21.15.96:0/2569588631 learned_addr learned my addr 172.21.15.96:0/2569588631 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:49.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 -- 172.21.15.96:0/2569588631 >> v1:172.21.15.96:6789/0 conn(0x7f66e013c8b0 legacy=0x7f66e006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 -- 172.21.15.96:0/2569588631 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0071bd0 con 0x7f66e013c4d0 2024-09-13T17:44:49.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66e5765640 1 --2- 172.21.15.96:0/2569588631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e006ff00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0097f0 tx=0x7f66dc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f877cc1d21045a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66d7fff640 1 -- 172.21.15.96:0/2569588631 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66dc0ab530 con 0x7f66e013c4d0 2024-09-13T17:44:49.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66d7fff640 1 -- 172.21.15.96:0/2569588631 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc0a7050 con 0x7f66e013c4d0 2024-09-13T17:44:49.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.071+0000 7f66d7fff640 1 -- 172.21.15.96:0/2569588631 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66dc0ab7e0 con 0x7f66e013c4d0 2024-09-13T17:44:49.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/2569588631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 msgr2=0x7f66e006ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 --2- 172.21.15.96:0/2569588631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e006ff00 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0097f0 tx=0x7f66dc01c3f0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/2569588631 shutdown_connections 2024-09-13T17:44:49.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 --2- 172.21.15.96:0/2569588631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e006ff00 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/2569588631 >> 172.21.15.96:0/2569588631 conn(0x7f66e0098a60 msgr2=0x7f66e009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/2569588631 shutdown_connections 2024-09-13T17:44:49.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/2569588631 wait complete. 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 Processor -- start 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- start start 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e009d300 con 0x7f66e013c4d0 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e5765640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e5765640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35114/0 (socket says 172.21.15.96:35114) 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e5765640 1 -- 172.21.15.96:0/1936638155 learned_addr learned my addr 172.21.15.96:0/1936638155 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e5765640 1 -- 172.21.15.96:0/1936638155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc0094d0 con 0x7f66e013c4d0 2024-09-13T17:44:49.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e5765640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01a080 tx=0x7f66dc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66dc0b6420 con 0x7f66e013c4d0 2024-09-13T17:44:49.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc0b6580 con 0x7f66e013c4d0 2024-09-13T17:44:49.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f66dc0b6840 con 0x7f66e013c4d0 2024-09-13T17:44:49.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e013f820 con 0x7f66e013c4d0 2024-09-13T17:44:49.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.075+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e013f540 con 0x7f66e013c4d0 2024-09-13T17:44:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e006ff00 con 0x7f66e013c4d0 2024-09-13T17:44:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f66dc0b2040 con 0x7f66e013c4d0 2024-09-13T17:44:49.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66d5ffb640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 0x7f66c4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 2957+0+0 (secure 0 0 0) 0x7f66dc0e1db0 con 0x7f66e013c4d0 2024-09-13T17:44:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66e4f64640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 0x7f66c4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:49.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.079+0000 7f66e4f64640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 0x7f66c4037b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66d0001570 tx=0x7f66d00079e0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:49.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.083+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f66dc0b69a0 con 0x7f66e013c4d0 2024-09-13T17:44:49.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.243+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f66e009e280 con 0x7f66e013c4d0 2024-09-13T17:44:49.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.915+0000 7f66d5ffb640 1 -- 172.21.15.96:0/1936638155 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v54) v1 ==== 170+0+0 (secure 0 0 0) 0x7f66dc0b0030 con 0x7f66e013c4d0 2024-09-13T17:44:49.921 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-13T17:44:49.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 msgr2=0x7f66c4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 0x7f66c4037b50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f66d0001570 tx=0x7f66d00079e0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 msgr2=0x7f66e009dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:49.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f66dc01a080 tx=0x7f66dc01ac90 comp rx=0 tx=0).stop 2024-09-13T17:44:49.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 shutdown_connections 2024-09-13T17:44:49.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f66c4035690 0x7f66c4037b50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 --2- 172.21.15.96:0/1936638155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66e013c4d0 0x7f66e009dd40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:49.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.923+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 >> 172.21.15.96:0/1936638155 conn(0x7f66e0098a60 msgr2=0x7f66e0135ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:49.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.927+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 shutdown_connections 2024-09-13T17:44:49.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:49.927+0000 7f66e79c9640 1 -- 172.21.15.96:0/1936638155 wait complete. 2024-09-13T17:44:49.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:44:50.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f776a9ce640 1 Processor -- start 2024-09-13T17:44:50.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f776a9ce640 1 -- start start 2024-09-13T17:44:50.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f776a9ce640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f77640a1470 0x7f77640a1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f776a9ce640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77640a2cb0 con 0x7f77640a1d90 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f776a9ce640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f776409de30 con 0x7f77640a1470 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f7763fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f77640a1470 0x7f77640a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f77637fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f77640a1d90 0x7f776413c290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43600/0 (socket says 172.21.15.96:43600) 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f77637fe640 1 -- 172.21.15.96:0/1246258589 learned_addr learned my addr 172.21.15.96:0/1246258589 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f7763fff640 1 -- 172.21.15.96:0/1246258589 >> v1:172.21.15.96:6789/0 conn(0x7f77640a1d90 legacy=0x7f776413c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f7763fff640 1 -- 172.21.15.96:0/1246258589 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776409db90 con 0x7f77640a1470 2024-09-13T17:44:50.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f7762ffd640 1 -- 172.21.15.96:0/1246258589 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2429432614 0 0) 0x7f7754001120 con 0x7f77640a1d90 2024-09-13T17:44:50.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.083+0000 7f7763fff640 1 --2- 172.21.15.96:0/1246258589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f77640a1850 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7758009740 tx=0x7f775801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd3ba9157adb8fd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7762ffd640 1 -- 172.21.15.96:0/1246258589 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77580ab530 con 0x7f77640a1470 2024-09-13T17:44:50.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7762ffd640 1 -- 172.21.15.96:0/1246258589 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77580a7050 con 0x7f77640a1470 2024-09-13T17:44:50.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7762ffd640 1 -- 172.21.15.96:0/1246258589 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77580ab7e0 con 0x7f77640a1470 2024-09-13T17:44:50.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/1246258589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 msgr2=0x7f77640a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 --2- 172.21.15.96:0/1246258589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f77640a1850 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7758009740 tx=0x7f775801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/1246258589 shutdown_connections 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 --2- 172.21.15.96:0/1246258589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f77640a1850 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/1246258589 >> 172.21.15.96:0/1246258589 conn(0x7f776406a150 msgr2=0x7f776406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/1246258589 shutdown_connections 2024-09-13T17:44:50.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/1246258589 wait complete. 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 Processor -- start 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- start start 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7764126a70 con 0x7f77640a1470 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7763fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7763fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35130/0 (socket says 172.21.15.96:35130) 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7763fff640 1 -- 172.21.15.96:0/838453301 learned_addr learned my addr 172.21.15.96:0/838453301 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7763fff640 1 -- 172.21.15.96:0/838453301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758009420 con 0x7f77640a1470 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7763fff640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f77580ae040 tx=0x7f7758019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77580b4540 con 0x7f77640a1470 2024-09-13T17:44:50.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77580b46a0 con 0x7f77640a1470 2024-09-13T17:44:50.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77640a22f0 con 0x7f77640a1470 2024-09-13T17:44:50.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77580b4980 con 0x7f77640a1470 2024-09-13T17:44:50.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.087+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7764125af0 con 0x7f77640a1470 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7764125850 con 0x7f77640a1470 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f77580b2040 con 0x7f77640a1470 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f7760ff9640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 0x7f7750037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 3339+0+0 (secure 0 0 0) 0x7f77580e1de0 con 0x7f77640a1470 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f77637fe640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 0x7f7750037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.091+0000 7f77637fe640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 0x7f7750037b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7754000c10 tx=0x7f7754001320 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.095+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f77580b4e60 con 0x7f77640a1470 2024-09-13T17:44:50.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.287+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7764123f90 con 0x7f77640a1470 2024-09-13T17:44:50.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.295+0000 7f7760ff9640 1 -- 172.21.15.96:0/838453301 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v32) v1 ==== 174+0+0 (secure 0 0 0) 0x7f77580b0030 con 0x7f77640a1470 2024-09-13T17:44:50.301 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 10 and data pool 11 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 msgr2=0x7f7750037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 0x7f7750037b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7754000c10 tx=0x7f7754001320 comp rx=0 tx=0).stop 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 msgr2=0x7f7764126530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f77580ae040 tx=0x7f7758019130 comp rx=0 tx=0).stop 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 shutdown_connections 2024-09-13T17:44:50.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f7750035690 0x7f7750037b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 --2- 172.21.15.96:0/838453301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77640a1470 0x7f7764126530 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 >> 172.21.15.96:0/838453301 conn(0x7f776406a150 msgr2=0x7f77640704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 shutdown_connections 2024-09-13T17:44:50.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.303+0000 7f776a9ce640 1 -- 172.21.15.96:0/838453301 wait complete. 2024-09-13T17:44:50.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-13T17:44:50.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:50.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc573f0640 1 Processor -- start 2024-09-13T17:44:50.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc573f0640 1 -- start start 2024-09-13T17:44:50.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc573f0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc50134e70 0x7efc5009e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc573f0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5009ed60 con 0x7efc50135250 2024-09-13T17:44:50.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc573f0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5009ddc0 con 0x7efc50134e70 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc5518c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efc50134e70 0x7efc5009e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc5498b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efc50135250 0x7efc50135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43604/0 (socket says 172.21.15.96:43604) 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc5498b640 1 -- 172.21.15.96:0/2857663738 learned_addr learned my addr 172.21.15.96:0/2857663738 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc5518c640 1 -- 172.21.15.96:0/2857663738 >> v1:172.21.15.96:6789/0 conn(0x7efc50135250 legacy=0x7efc50135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc47fff640 1 -- 172.21.15.96:0/2857663738 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3222403639 0 0) 0x7efc40001120 con 0x7efc50135250 2024-09-13T17:44:50.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.459+0000 7efc5518c640 1 -- 172.21.15.96:0/2857663738 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500a68b0 con 0x7efc50134e70 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc5518c640 1 --2- 172.21.15.96:0/2857663738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5009e820 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009740 tx=0x7efc4c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2884844d863fddc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc47fff640 1 -- 172.21.15.96:0/2857663738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0ab530 con 0x7efc50134e70 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc47fff640 1 -- 172.21.15.96:0/2857663738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c0a7050 con 0x7efc50134e70 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc47fff640 1 -- 172.21.15.96:0/2857663738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0ab7e0 con 0x7efc50134e70 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- 172.21.15.96:0/2857663738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 msgr2=0x7efc5009e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 --2- 172.21.15.96:0/2857663738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5009e820 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009740 tx=0x7efc4c01c420 comp rx=0 tx=0).stop 2024-09-13T17:44:50.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- 172.21.15.96:0/2857663738 shutdown_connections 2024-09-13T17:44:50.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 --2- 172.21.15.96:0/2857663738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5009e820 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- 172.21.15.96:0/2857663738 >> 172.21.15.96:0/2857663738 conn(0x7efc50098a60 msgr2=0x7efc5009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- 172.21.15.96:0/2857663738 shutdown_connections 2024-09-13T17:44:50.470 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- 172.21.15.96:0/2857663738 wait complete. 2024-09-13T17:44:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 Processor -- start 2024-09-13T17:44:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- start start 2024-09-13T17:44:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.463+0000 7efc573f0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc500a4f90 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5518c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5518c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35156/0 (socket says 172.21.15.96:35156) 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5518c640 1 -- 172.21.15.96:0/1976758263 learned_addr learned my addr 172.21.15.96:0/1976758263 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5518c640 1 -- 172.21.15.96:0/1976758263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c009420 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5518c640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7efc4c019030 tx=0x7efc4c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0b7450 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4c0b75b0 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50070800 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efc4c0b7890 con 0x7efc50134e70 2024-09-13T17:44:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc500711e0 con 0x7efc50134e70 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc501368b0 con 0x7efc50134e70 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7efc4c0b79f0 con 0x7efc50134e70 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 0x7efc34037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 3439+0+0 (secure 0 0 0) 0x7efc4c0e2530 con 0x7efc50134e70 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5498b640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 0x7efc34037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.467+0000 7efc5498b640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 0x7efc34037b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c10 tx=0x7efc40001320 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.471+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efc4c0c74e0 con 0x7efc50134e70 2024-09-13T17:44:50.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.631+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc50071530 con 0x7efc50134e70 2024-09-13T17:44:50.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc45ffb640 1 -- 172.21.15.96:0/1976758263 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+10507 (secure 0 0 0) 0x7efc4c0bb090 con 0x7efc50134e70 2024-09-13T17:44:50.639 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:50.639 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":55,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:50.294219+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:50.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 msgr2=0x7efc34037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 0x7efc34037b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7efc40000c10 tx=0x7efc40001320 comp rx=0 tx=0).stop 2024-09-13T17:44:50.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 msgr2=0x7efc5006f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7efc4c019030 tx=0x7efc4c01ac60 comp rx=0 tx=0).stop 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 shutdown_connections 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7efc34035690 0x7efc34037b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 --2- 172.21.15.96:0/1976758263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc50134e70 0x7efc5006f6d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 >> 172.21.15.96:0/1976758263 conn(0x7efc50098a60 msgr2=0x7efc50098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 shutdown_connections 2024-09-13T17:44:50.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.635+0000 7efc573f0640 1 -- 172.21.15.96:0/1976758263 wait complete. 2024-09-13T17:44:50.658 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b9106640 1 Processor -- start 2024-09-13T17:44:50.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b9106640 1 -- start start 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b9106640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc3b4134e70 0x7fc3b409e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b9106640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b409ed60 con 0x7fc3b4135250 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b9106640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b409ddc0 con 0x7fc3b4134e70 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b2d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc3b4134e70 0x7fc3b409e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b259c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc3b4135250 0x7fc3b4135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43620/0 (socket says 172.21.15.96:43620) 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.767+0000 7fc3b259c640 1 -- 172.21.15.96:0/3215065501 learned_addr learned my addr 172.21.15.96:0/3215065501 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b2d9d640 1 -- 172.21.15.96:0/3215065501 >> v1:172.21.15.96:6789/0 conn(0x7fc3b4135250 legacy=0x7fc3b4135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b2d9d640 1 -- 172.21.15.96:0/3215065501 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b40a68b0 con 0x7fc3b4134e70 2024-09-13T17:44:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b2d9d640 1 --2- 172.21.15.96:0/3215065501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b409e820 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009740 tx=0x7fc3a401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c82ebe25c96800 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b1d9b640 1 -- 172.21.15.96:0/3215065501 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a40ab530 con 0x7fc3b4134e70 2024-09-13T17:44:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b1d9b640 1 -- 172.21.15.96:0/3215065501 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40a7050 con 0x7fc3b4134e70 2024-09-13T17:44:50.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b1d9b640 1 -- 172.21.15.96:0/3215065501 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a40ab7e0 con 0x7fc3b4134e70 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 -- 172.21.15.96:0/3215065501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 msgr2=0x7fc3b409e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 --2- 172.21.15.96:0/3215065501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b409e820 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4009740 tx=0x7fc3a401c420 comp rx=0 tx=0).stop 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 -- 172.21.15.96:0/3215065501 shutdown_connections 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 --2- 172.21.15.96:0/3215065501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b409e820 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 -- 172.21.15.96:0/3215065501 >> 172.21.15.96:0/3215065501 conn(0x7fc3b4098a60 msgr2=0x7fc3b409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 -- 172.21.15.96:0/3215065501 shutdown_connections 2024-09-13T17:44:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.771+0000 7fc3b9106640 1 -- 172.21.15.96:0/3215065501 wait complete. 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 Processor -- start 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 -- start start 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b4070ba0 con 0x7fc3b4134e70 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35170/0 (socket says 172.21.15.96:35170) 2024-09-13T17:44:50.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b2d9d640 1 -- 172.21.15.96:0/917321455 learned_addr learned my addr 172.21.15.96:0/917321455 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b2d9d640 1 -- 172.21.15.96:0/917321455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4009420 con 0x7fc3b4134e70 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b2d9d640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ae040 tx=0x7fc3a40191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a40abea0 con 0x7fc3b4134e70 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a40b4410 con 0x7fc3b4134e70 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b40a3f20 con 0x7fc3b4134e70 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a40b46f0 con 0x7fc3b4134e70 2024-09-13T17:44:50.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b40a37a0 con 0x7fc3b4134e70 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b41368b0 con 0x7fc3b4134e70 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fc3a40b2040 con 0x7fc3b4134e70 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 0x7fc394037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fc3a40e1d70 con 0x7fc3b4134e70 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b259c640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 0x7fc394037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:50.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.775+0000 7fc3b259c640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 0x7fc394037b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc39c000f70 tx=0x7fc39c007a90 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:50.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.779+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc3a40bc3b0 con 0x7fc3b4134e70 2024-09-13T17:44:50.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.935+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3b406a1b0 con 0x7fc3b4134e70 2024-09-13T17:44:50.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3ab7fe640 1 -- 172.21.15.96:0/917321455 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+10507 (secure 0 0 0) 0x7fc3a40c2020 con 0x7fc3b4134e70 2024-09-13T17:44:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:50.943 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":55,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:50.294219+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 msgr2=0x7fc394037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 0x7fc394037b50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc39c000f70 tx=0x7fc39c007a90 comp rx=0 tx=0).stop 2024-09-13T17:44:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 msgr2=0x7fc3b406f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fc3a40ae040 tx=0x7fc3a40191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 shutdown_connections 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fc394035690 0x7fc394037b50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 --2- 172.21.15.96:0/917321455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3b4134e70 0x7fc3b406f6d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 >> 172.21.15.96:0/917321455 conn(0x7fc3b4098a60 msgr2=0x7fc3b4131da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 shutdown_connections 2024-09-13T17:44:50.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:50.939+0000 7fc3b9106640 1 -- 172.21.15.96:0/917321455 wait complete. 2024-09-13T17:44:50.960 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:44:51.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.075+0000 7f14476a2640 1 Processor -- start 2024-09-13T17:44:51.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.075+0000 7f14476a2640 1 -- start start 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f144013af50 0x7f144013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1440138050 con 0x7f144009d690 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f144009c410 con 0x7f144013af50 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1444c3d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f144013af50 0x7f144013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f144543e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f144009d690 0x7f1440138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43622/0 (socket says 172.21.15.96:43622) 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f144543e640 1 -- 172.21.15.96:0/2628160853 learned_addr learned my addr 172.21.15.96:0/2628160853 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1444c3d640 1 --2- 172.21.15.96:0/2628160853 >> v2:172.21.15.96:3300/0 conn(0x7f144013af50 0x7f144013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35178/0 (socket says 172.21.15.96:35178) 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1444c3d640 1 -- 172.21.15.96:0/2628160853 >> v1:172.21.15.96:6789/0 conn(0x7f144009d690 legacy=0x7f1440138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:51.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1444c3d640 1 -- 172.21.15.96:0/2628160853 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f144009cba0 con 0x7f144013af50 2024-09-13T17:44:51.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1444c3d640 1 --2- 172.21.15.96:0/2628160853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144013af50 0x7f144013d340 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f143c009740 tx=0x7f143c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d462dbc7fb80aa server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:51.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1437fff640 1 -- 172.21.15.96:0/2628160853 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f143c0a8840 con 0x7f144013af50 2024-09-13T17:44:51.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1437fff640 1 -- 172.21.15.96:0/2628160853 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143c0a89a0 con 0x7f144013af50 2024-09-13T17:44:51.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f1437fff640 1 -- 172.21.15.96:0/2628160853 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f143c0a8c60 con 0x7f144013af50 2024-09-13T17:44:51.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- 172.21.15.96:0/2628160853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144013af50 msgr2=0x7f144013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:51.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 --2- 172.21.15.96:0/2628160853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144013af50 0x7f144013d340 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f143c009740 tx=0x7f143c01ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:51.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- 172.21.15.96:0/2628160853 shutdown_connections 2024-09-13T17:44:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 --2- 172.21.15.96:0/2628160853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144013af50 0x7f144013d340 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- 172.21.15.96:0/2628160853 >> 172.21.15.96:0/2628160853 conn(0x7f144006bc60 msgr2=0x7f144006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- 172.21.15.96:0/2628160853 shutdown_connections 2024-09-13T17:44:51.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.079+0000 7f14476a2640 1 -- 172.21.15.96:0/2628160853 wait complete. 2024-09-13T17:44:51.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 Processor -- start 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 -- start start 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14400a4f90 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f144543e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f144543e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35188/0 (socket says 172.21.15.96:35188) 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f144543e640 1 -- 172.21.15.96:0/1240829856 learned_addr learned my addr 172.21.15.96:0/1240829856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f144543e640 1 -- 172.21.15.96:0/1240829856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c009420 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f144543e640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f143000c380 tx=0x7f143000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1430005c20 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430005d80 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1430007200 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14400a2c20 con 0x7f144009d690 2024-09-13T17:44:51.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14401373b0 con 0x7f144009d690 2024-09-13T17:44:51.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400a3f70 con 0x7f144009d690 2024-09-13T17:44:51.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f1430007360 con 0x7f144009d690 2024-09-13T17:44:51.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 0x7f1424037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:51.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f1430046cb0 con 0x7f144009d690 2024-09-13T17:44:51.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1444c3d640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 0x7f1424037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:51.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.083+0000 7f1444c3d640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 0x7f1424037b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f143c0b2000 tx=0x7f143c0aa070 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:51.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.087+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f14400a3f70 con 0x7f144009d690 2024-09-13T17:44:51.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:51.283+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1440138050 con 0x7f144009d690 2024-09-13T17:44:52.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.059+0000 7f1435ffb640 1 -- 172.21.15.96:0/1240829856 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1440138050 con 0x7f144009d690 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 msgr2=0x7f1424037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 0x7f1424037b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f143c0b2000 tx=0x7f143c0aa070 comp rx=0 tx=0).stop 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 msgr2=0x7f14401c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f143000c380 tx=0x7f143000c840 comp rx=0 tx=0).stop 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 shutdown_connections 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f1424035690 0x7f1424037b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:52.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 --2- 172.21.15.96:0/1240829856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f144009d690 0x7f14401c9820 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:52.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 >> 172.21.15.96:0/1240829856 conn(0x7f144006bc60 msgr2=0x7f1440131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:52.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 shutdown_connections 2024-09-13T17:44:52.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.063+0000 7f14476a2640 1 -- 172.21.15.96:0/1240829856 wait complete. 2024-09-13T17:44:52.089 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:44:52.089 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:44:52.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.211+0000 7ff8a914d640 1 Processor -- start 2024-09-13T17:44:52.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.211+0000 7ff8a914d640 1 -- start start 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a914d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff8a406f6d0 0x7ff8a4134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a914d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a406e130 con 0x7ff8a409dc20 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a914d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a40a6750 con 0x7ff8a406f6d0 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a2d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff8a406f6d0 0x7ff8a4134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff89bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff8a409dc20 0x7ff8a4134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:43626/0 (socket says 172.21.15.96:43626) 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff89bfff640 1 -- 172.21.15.96:0/413855528 learned_addr learned my addr 172.21.15.96:0/413855528 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a2d9d640 1 -- 172.21.15.96:0/413855528 >> v1:172.21.15.96:6789/0 conn(0x7ff8a409dc20 legacy=0x7ff8a4134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a259c640 1 -- 172.21.15.96:0/413855528 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552985157 0 0) 0x7ff8a406e130 con 0x7ff8a409dc20 2024-09-13T17:44:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a2d9d640 1 -- 172.21.15.96:0/413855528 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40712e0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a2d9d640 1 --2- 172.21.15.96:0/413855528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a4134ab0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff894009740 tx=0x7ff89401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e02850536de358 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a259c640 1 -- 172.21.15.96:0/413855528 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8940ab400 con 0x7ff8a406f6d0 2024-09-13T17:44:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a259c640 1 -- 172.21.15.96:0/413855528 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940a7050 con 0x7ff8a406f6d0 2024-09-13T17:44:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a259c640 1 -- 172.21.15.96:0/413855528 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8940ab6b0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a914d640 1 -- 172.21.15.96:0/413855528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 msgr2=0x7ff8a4134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.215+0000 7ff8a914d640 1 --2- 172.21.15.96:0/413855528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a4134ab0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff894009740 tx=0x7ff89401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/413855528 shutdown_connections 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 --2- 172.21.15.96:0/413855528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a4134ab0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/413855528 >> 172.21.15.96:0/413855528 conn(0x7ff8a4098a60 msgr2=0x7ff8a409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:52.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/413855528 shutdown_connections 2024-09-13T17:44:52.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/413855528 wait complete. 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 Processor -- start 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- start start 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41216d0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35208/0 (socket says 172.21.15.96:35208) 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a2d9d640 1 -- 172.21.15.96:0/3910632747 learned_addr learned my addr 172.21.15.96:0/3910632747 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:52.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a2d9d640 1 -- 172.21.15.96:0/3910632747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009420 con 0x7ff8a406f6d0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a2d9d640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff894019fc0 tx=0x7ff89401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8940b7450 con 0x7ff8a406f6d0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8940b75b0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a4120780 con 0x7ff8a406f6d0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff8940b7850 con 0x7ff8a406f6d0 2024-09-13T17:44:52.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a411e8d0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7ff8940b79b0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a411ec20 con 0x7ff8a406f6d0 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 0x7ff884037b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff8940e2500 con 0x7ff8a406f6d0 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.219+0000 7ff89bfff640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 0x7ff884037b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.223+0000 7ff89bfff640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 0x7ff884037b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff88c007fd0 tx=0x7ff88c007940 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:52.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.227+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff8940c78f0 con 0x7ff8a406f6d0 2024-09-13T17:44:52.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:52.423+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff8a40a55e0 con 0x7ff8a406f6d0 2024-09-13T17:44:53.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.299+0000 7ff89b7fe640 1 -- 172.21.15.96:0/3910632747 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8940bb0c0 con 0x7ff8a406f6d0 2024-09-13T17:44:53.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 msgr2=0x7ff884037b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 0x7ff884037b00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff88c007fd0 tx=0x7ff88c007940 comp rx=0 tx=0).stop 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 msgr2=0x7ff8a411cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7ff894019fc0 tx=0x7ff89401abd0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 shutdown_connections 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff884035640 0x7ff884037b00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 --2- 172.21.15.96:0/3910632747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8a406f6d0 0x7ff8a411cd30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 >> 172.21.15.96:0/3910632747 conn(0x7ff8a4098a60 msgr2=0x7ff8a40a5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 shutdown_connections 2024-09-13T17:44:53.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.303+0000 7ff8a914d640 1 -- 172.21.15.96:0/3910632747 wait complete. 2024-09-13T17:44:53.330 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:44:53.331 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:53.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.447+0000 7ff6335be640 1 Processor -- start 2024-09-13T17:44:53.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.447+0000 7ff6335be640 1 -- start start 2024-09-13T17:44:53.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff62c09e820 0x7ff62c13ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62c0a68b0 con 0x7ff62c134e70 2024-09-13T17:44:53.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff62c09ddc0 con 0x7ff62c09e820 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff63135a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff62c134e70 0x7ff62c135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59230/0 (socket says 172.21.15.96:59230) 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff630b59640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff62c09e820 0x7ff62c13ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff63135a640 1 -- 172.21.15.96:0/4022094516 learned_addr learned my addr 172.21.15.96:0/4022094516 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff630b59640 1 -- 172.21.15.96:0/4022094516 >> v1:172.21.15.96:6789/0 conn(0x7ff62c134e70 legacy=0x7ff62c135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff630b59640 1 -- 172.21.15.96:0/4022094516 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c136970 con 0x7ff62c09e820 2024-09-13T17:44:53.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff62bfff640 1 -- 172.21.15.96:0/4022094516 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830331024 0 0) 0x7ff62c0a68b0 con 0x7ff62c134e70 2024-09-13T17:44:53.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff630b59640 1 --2- 172.21.15.96:0/4022094516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c13ae50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff620009740 tx=0x7ff62001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ee630a1f9911c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff62bfff640 1 -- 172.21.15.96:0/4022094516 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6200ab530 con 0x7ff62c09e820 2024-09-13T17:44:53.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff62bfff640 1 -- 172.21.15.96:0/4022094516 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6200a7050 con 0x7ff62c09e820 2024-09-13T17:44:53.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff62bfff640 1 -- 172.21.15.96:0/4022094516 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6200ab7e0 con 0x7ff62c09e820 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- 172.21.15.96:0/4022094516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 msgr2=0x7ff62c13ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 --2- 172.21.15.96:0/4022094516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c13ae50 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff620009740 tx=0x7ff62001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- 172.21.15.96:0/4022094516 shutdown_connections 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 --2- 172.21.15.96:0/4022094516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c13ae50 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- 172.21.15.96:0/4022094516 >> 172.21.15.96:0/4022094516 conn(0x7ff62c098a60 msgr2=0x7ff62c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- 172.21.15.96:0/4022094516 shutdown_connections 2024-09-13T17:44:53.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.451+0000 7ff6335be640 1 -- 172.21.15.96:0/4022094516 wait complete. 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 Processor -- start 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 -- start start 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c11ca40 con 0x7ff62c09e820 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff63135a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff63135a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40504/0 (socket says 172.21.15.96:40504) 2024-09-13T17:44:53.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff63135a640 1 -- 172.21.15.96:0/2945958816 learned_addr learned my addr 172.21.15.96:0/2945958816 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:53.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff63135a640 1 -- 172.21.15.96:0/2945958816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620009420 con 0x7ff62c09e820 2024-09-13T17:44:53.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff63135a640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c840 tx=0x7ff62400cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff62400a660 con 0x7ff62c09e820 2024-09-13T17:44:53.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62400a7e0 con 0x7ff62c09e820 2024-09-13T17:44:53.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c11d6e0 con 0x7ff62c09e820 2024-09-13T17:44:53.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff62400aac0 con 0x7ff62c09e820 2024-09-13T17:44:53.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c13a8f0 con 0x7ff62c09e820 2024-09-13T17:44:53.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7ff62400ac20 con 0x7ff62c09e820 2024-09-13T17:44:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c0a45e0 con 0x7ff62c09e820 2024-09-13T17:44:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 0x7ff618037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff6240473e0 con 0x7ff62c09e820 2024-09-13T17:44:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff630b59640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 0x7ff618037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.455+0000 7ff630b59640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 0x7ff618037b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ad000 tx=0x7ff620008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.463+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff62401a2c0 con 0x7ff62c09e820 2024-09-13T17:44:53.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.655+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff62c13a430 con 0x7ff62c09e820 2024-09-13T17:44:53.661 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.655+0000 7ff629ffb640 1 -- 172.21.15.96:0/2945958816 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7ff624018070 con 0x7ff62c09e820 2024-09-13T17:44:53.661 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:53.662 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:44:53.662 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:44:53.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 msgr2=0x7ff618037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 0x7ff618037b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff6200ad000 tx=0x7ff620008040 comp rx=0 tx=0).stop 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 msgr2=0x7ff62c0a26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c840 tx=0x7ff62400cd00 comp rx=0 tx=0).stop 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 shutdown_connections 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7ff618035690 0x7ff618037b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 --2- 172.21.15.96:0/2945958816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff62c09e820 0x7ff62c0a26c0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 >> 172.21.15.96:0/2945958816 conn(0x7ff62c098a60 msgr2=0x7ff62c133d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 shutdown_connections 2024-09-13T17:44:53.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.659+0000 7ff6335be640 1 -- 172.21.15.96:0/2945958816 wait complete. 2024-09-13T17:44:53.681 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:44:53.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.795+0000 7f2bff383640 1 Processor -- start 2024-09-13T17:44:53.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.795+0000 7f2bff383640 1 -- start start 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bf809dc20 0x7f2bf81351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf806e130 con 0x7f2bf806f6d0 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80a6750 con 0x7f2bf809dc20 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfd11f640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2bf806f6d0 0x7f2bf8134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59234/0 (socket says 172.21.15.96:59234) 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfc91e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2bf809dc20 0x7f2bf81351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfd11f640 1 -- 172.21.15.96:0/454482318 learned_addr learned my addr 172.21.15.96:0/454482318 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:53.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfc91e640 1 -- 172.21.15.96:0/454482318 >> v1:172.21.15.96:6789/0 conn(0x7f2bf806f6d0 legacy=0x7f2bf8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfc91e640 1 -- 172.21.15.96:0/454482318 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80712e0 con 0x7f2bf809dc20 2024-09-13T17:44:53.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bfc91e640 1 --2- 172.21.15.96:0/454482318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf809dc20 0x7f2bf81351c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009740 tx=0x7f2be801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafbf9bc9da506b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2beffff640 1 -- 172.21.15.96:0/454482318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be80ab400 con 0x7f2bf809dc20 2024-09-13T17:44:53.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2beffff640 1 -- 172.21.15.96:0/454482318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be80a7050 con 0x7f2bf809dc20 2024-09-13T17:44:53.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2beffff640 1 -- 172.21.15.96:0/454482318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2be80ab6b0 con 0x7f2bf809dc20 2024-09-13T17:44:53.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- 172.21.15.96:0/454482318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf809dc20 msgr2=0x7f2bf81351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 --2- 172.21.15.96:0/454482318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf809dc20 0x7f2bf81351c0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009740 tx=0x7f2be801c390 comp rx=0 tx=0).stop 2024-09-13T17:44:53.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- 172.21.15.96:0/454482318 shutdown_connections 2024-09-13T17:44:53.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 --2- 172.21.15.96:0/454482318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf809dc20 0x7f2bf81351c0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- 172.21.15.96:0/454482318 >> 172.21.15.96:0/454482318 conn(0x7f2bf8098a60 msgr2=0x7f2bf809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:53.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- 172.21.15.96:0/454482318 shutdown_connections 2024-09-13T17:44:53.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.799+0000 7f2bff383640 1 -- 172.21.15.96:0/454482318 wait complete. 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 Processor -- start 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 -- start start 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80a5750 con 0x7f2bf806f6d0 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfd11f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfd11f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40518/0 (socket says 172.21.15.96:40518) 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfd11f640 1 -- 172.21.15.96:0/3338902571 learned_addr learned my addr 172.21.15.96:0/3338902571 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfd11f640 1 -- 172.21.15.96:0/3338902571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be8009420 con 0x7f2bf806f6d0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfd11f640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c480 tx=0x7f2bf400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf4005c20 con 0x7f2bf806f6d0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf4005d80 con 0x7f2bf806f6d0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8139590 con 0x7f2bf806f6d0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bf40071a0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf80a53b0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f2bf4007300 con 0x7f2bf806f6d0 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf80a37e0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 0x7f2bdc037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f2bf4046fb0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfc91e640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 0x7f2bdc037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:53.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.803+0000 7f2bfc91e640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 0x7f2bdc037b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ad000 tx=0x7f2be8008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:53.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.807+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bf40075b0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.967+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bf8134ab0 con 0x7f2bf806f6d0 2024-09-13T17:44:53.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bedffb640 1 -- 172.21.15.96:0/3338902571 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+10494 (secure 0 0 0) 0x7f2bf4017030 con 0x7f2bf806f6d0 2024-09-13T17:44:53.975 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:53.975 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":56,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:44:51.296024+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 msgr2=0x7f2bdc037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 0x7f2bdc037b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2be80ad000 tx=0x7f2be8008040 comp rx=0 tx=0).stop 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 msgr2=0x7f2bf80a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c480 tx=0x7f2bf400c940 comp rx=0 tx=0).stop 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 shutdown_connections 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2bdc035690 0x7f2bdc037b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 --2- 172.21.15.96:0/3338902571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bf806f6d0 0x7f2bf80a18e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 >> 172.21.15.96:0/3338902571 conn(0x7f2bf8098a60 msgr2=0x7f2bf81397a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:53.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 shutdown_connections 2024-09-13T17:44:53.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:53.971+0000 7f2bff383640 1 -- 172.21.15.96:0/3338902571 wait complete. 2024-09-13T17:44:53.994 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:44:53.994 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:54.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.103+0000 7fbbf7cd8640 1 Processor -- start 2024-09-13T17:44:54.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf7cd8640 1 -- start start 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf7cd8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf009e820 0x7fbbf013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf7cd8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00a68b0 con 0x7fbbf0134e70 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf7cd8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf009ddc0 con 0x7fbbf009e820 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf009e820 0x7fbbf013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbbf009e820 0x7fbbf013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40534/0 (socket says 172.21.15.96:40534) 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 -- 172.21.15.96:0/2715257519 learned_addr learned my addr 172.21.15.96:0/2715257519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:54.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5a74640 1 --1- 172.21.15.96:0/2715257519 >> v1:172.21.15.96:6789/0 conn(0x7fbbf0134e70 0x7fbbf0135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59236/0 (socket says 172.21.15.96:59236) 2024-09-13T17:44:54.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 -- 172.21.15.96:0/2715257519 >> v1:172.21.15.96:6789/0 conn(0x7fbbf0134e70 legacy=0x7fbbf0135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 -- 172.21.15.96:0/2715257519 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0136970 con 0x7fbbf009e820 2024-09-13T17:44:54.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf4a72640 1 -- 172.21.15.96:0/2715257519 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1766243124 0 0) 0x7fbbf00a68b0 con 0x7fbbf0134e70 2024-09-13T17:44:54.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf5273640 1 --2- 172.21.15.96:0/2715257519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf013ae50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb33e5424b546c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf4a72640 1 -- 172.21.15.96:0/2715257519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec009200 con 0x7fbbf009e820 2024-09-13T17:44:54.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf4a72640 1 -- 172.21.15.96:0/2715257519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec0a5750 con 0x7fbbf009e820 2024-09-13T17:44:54.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.107+0000 7fbbf4a72640 1 -- 172.21.15.96:0/2715257519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbec0a5a10 con 0x7fbbf009e820 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/2715257519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 msgr2=0x7fbbf013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/2715257519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf013ae50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fbbec009e50 tx=0x7fbbec01ca30 comp rx=0 tx=0).stop 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/2715257519 shutdown_connections 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/2715257519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf013ae50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/2715257519 >> 172.21.15.96:0/2715257519 conn(0x7fbbf0098a60 msgr2=0x7fbbf009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:54.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/2715257519 shutdown_connections 2024-09-13T17:44:54.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/2715257519 wait complete. 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 Processor -- start 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- start start 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00a1d10 con 0x7fbbf009e820 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5a74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5a74640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40546/0 (socket says 172.21.15.96:40546) 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5a74640 1 -- 172.21.15.96:0/375429557 learned_addr learned my addr 172.21.15.96:0/375429557 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5a74640 1 -- 172.21.15.96:0/375429557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbec009b30 con 0x7fbbf009e820 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5a74640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c420 tx=0x7fbbe000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe00074d0 con 0x7fbbf009e820 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0007630 con 0x7fbbf009e820 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf0132fe0 con 0x7fbbf009e820 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbbe0004960 con 0x7fbbf009e820 2024-09-13T17:44:54.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf00a0d30 con 0x7fbbf009e820 2024-09-13T17:44:54.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf01c9db0 con 0x7fbbf009e820 2024-09-13T17:44:54.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fbbe0004ac0 con 0x7fbbf009e820 2024-09-13T17:44:54.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 0x7fbbd4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fbbe0047090 con 0x7fbbf009e820 2024-09-13T17:44:54.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.111+0000 7fbbf5273640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 0x7fbbd4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.115+0000 7fbbf5273640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 0x7fbbd4037b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0ae000 tx=0x7fbbec0a7070 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.119+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbbe001acd0 con 0x7fbbf009e820 2024-09-13T17:44:54.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.275+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 --> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbf00a1e50 con 0x7fbbd4035690 2024-09-13T17:44:54.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.291+0000 7fbbe67fc640 1 -- 172.21.15.96:0/375429557 <== mgr.4986 v2:172.21.15.96:6824/3506364525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250052 (secure 0 0 0) 0x7fbbf00a1e50 con 0x7fbbd4035690 2024-09-13T17:44:54.300 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:54.302 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:44:54.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 msgr2=0x7fbbd4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 0x7fbbd4037b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbbec0ae000 tx=0x7fbbec0a7070 comp rx=0 tx=0).stop 2024-09-13T17:44:54.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 msgr2=0x7fbbf00a28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fbbe000c420 tx=0x7fbbe000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 shutdown_connections 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbbd4035690 0x7fbbd4037b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 --2- 172.21.15.96:0/375429557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbbf009e820 0x7fbbf00a28e0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 >> 172.21.15.96:0/375429557 conn(0x7fbbf0098a60 msgr2=0x7fbbf01323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 shutdown_connections 2024-09-13T17:44:54.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.295+0000 7fbbf7cd8640 1 -- 172.21.15.96:0/375429557 wait complete. 2024-09-13T17:44:54.318 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-13T17:44:53.596693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":261,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96152,"kb_used_data":2780,"kb_used_omap":4,"kb_used_meta":93307,"kb_avail":283019368,"statfs":{"total":289910292480,"available":289811832832,"internally_reserved":0,"allocated":2846720,"data_stored":1806212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":95546707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.691718"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348595+0000","last_change":"2024-09-13T17:44:50.342910+0000","last_active":"2024-09-13T17:44:50.348595+0000","last_peered":"2024-09-13T17:44:50.348595+0000","last_clean":"2024-09-13T17:44:50.348595+0000","last_became_active":"2024-09-13T17:44:50.342120+0000","last_became_peered":"2024-09-13T17:44:50.342120+0000","last_unstale":"2024-09-13T17:44:50.348595+0000","last_undegraded":"2024-09-13T17:44:50.348595+0000","last_fullsized":"2024-09-13T17:44:50.348595+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:11.433421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335521+0000","last_change":"2024-09-13T17:44:50.330106+0000","last_active":"2024-09-13T17:44:50.335521+0000","last_peered":"2024-09-13T17:44:50.335521+0000","last_clean":"2024-09-13T17:44:50.335521+0000","last_became_active":"2024-09-13T17:44:50.328575+0000","last_became_peered":"2024-09-13T17:44:50.328575+0000","last_unstale":"2024-09-13T17:44:50.335521+0000","last_undegraded":"2024-09-13T17:44:50.335521+0000","last_fullsized":"2024-09-13T17:44:50.335521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:30.164165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302636+0000","last_change":"2024-09-13T17:44:50.328043+0000","last_active":"2024-09-13T17:44:51.302636+0000","last_peered":"2024-09-13T17:44:51.302636+0000","last_clean":"2024-09-13T17:44:51.302636+0000","last_became_active":"2024-09-13T17:44:50.323021+0000","last_became_peered":"2024-09-13T17:44:50.323021+0000","last_unstale":"2024-09-13T17:44:51.302636+0000","last_undegraded":"2024-09-13T17:44:51.302636+0000","last_fullsized":"2024-09-13T17:44:51.302636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:20.503150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349282+0000","last_change":"2024-09-13T17:44:50.344770+0000","last_active":"2024-09-13T17:44:50.349282+0000","last_peered":"2024-09-13T17:44:50.349282+0000","last_clean":"2024-09-13T17:44:50.349282+0000","last_became_active":"2024-09-13T17:44:50.344481+0000","last_became_peered":"2024-09-13T17:44:50.344481+0000","last_unstale":"2024-09-13T17:44:50.349282+0000","last_undegraded":"2024-09-13T17:44:50.349282+0000","last_fullsized":"2024-09-13T17:44:50.349282+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:38.778989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301481+0000","last_change":"2024-09-13T17:44:50.320652+0000","last_active":"2024-09-13T17:44:51.301481+0000","last_peered":"2024-09-13T17:44:51.301481+0000","last_clean":"2024-09-13T17:44:51.301481+0000","last_became_active":"2024-09-13T17:44:50.320149+0000","last_became_peered":"2024-09-13T17:44:50.320149+0000","last_unstale":"2024-09-13T17:44:51.301481+0000","last_undegraded":"2024-09-13T17:44:51.301481+0000","last_fullsized":"2024-09-13T17:44:51.301481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:03.443486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300909+0000","last_change":"2024-09-13T17:44:50.324956+0000","last_active":"2024-09-13T17:44:51.300909+0000","last_peered":"2024-09-13T17:44:51.300909+0000","last_clean":"2024-09-13T17:44:51.300909+0000","last_became_active":"2024-09-13T17:44:50.319923+0000","last_became_peered":"2024-09-13T17:44:50.319923+0000","last_unstale":"2024-09-13T17:44:51.300909+0000","last_undegraded":"2024-09-13T17:44:51.300909+0000","last_fullsized":"2024-09-13T17:44:51.300909+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:44.363201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348697+0000","last_change":"2024-09-13T17:44:50.345091+0000","last_active":"2024-09-13T17:44:50.348697+0000","last_peered":"2024-09-13T17:44:50.348697+0000","last_clean":"2024-09-13T17:44:50.348697+0000","last_became_active":"2024-09-13T17:44:50.344845+0000","last_became_peered":"2024-09-13T17:44:50.344845+0000","last_unstale":"2024-09-13T17:44:50.348697+0000","last_undegraded":"2024-09-13T17:44:50.348697+0000","last_fullsized":"2024-09-13T17:44:50.348697+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:31.116346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335983+0000","last_change":"2024-09-13T17:44:50.332460+0000","last_active":"2024-09-13T17:44:50.335983+0000","last_peered":"2024-09-13T17:44:50.335983+0000","last_clean":"2024-09-13T17:44:50.335983+0000","last_became_active":"2024-09-13T17:44:50.332067+0000","last_became_peered":"2024-09-13T17:44:50.332067+0000","last_unstale":"2024-09-13T17:44:50.335983+0000","last_undegraded":"2024-09-13T17:44:50.335983+0000","last_fullsized":"2024-09-13T17:44:50.335983+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:13:01.246561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301991+0000","last_change":"2024-09-13T17:44:50.317608+0000","last_active":"2024-09-13T17:44:51.301991+0000","last_peered":"2024-09-13T17:44:51.301991+0000","last_clean":"2024-09-13T17:44:51.301991+0000","last_became_active":"2024-09-13T17:44:50.317343+0000","last_became_peered":"2024-09-13T17:44:50.317343+0000","last_unstale":"2024-09-13T17:44:51.301991+0000","last_undegraded":"2024-09-13T17:44:51.301991+0000","last_fullsized":"2024-09-13T17:44:51.301991+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:37.835126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348620+0000","last_change":"2024-09-13T17:44:50.344641+0000","last_active":"2024-09-13T17:44:50.348620+0000","last_peered":"2024-09-13T17:44:50.348620+0000","last_clean":"2024-09-13T17:44:50.348620+0000","last_became_active":"2024-09-13T17:44:50.343162+0000","last_became_peered":"2024-09-13T17:44:50.343162+0000","last_unstale":"2024-09-13T17:44:50.348620+0000","last_undegraded":"2024-09-13T17:44:50.348620+0000","last_fullsized":"2024-09-13T17:44:50.348620+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:33.970616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346093+0000","last_change":"2024-09-13T17:44:50.341688+0000","last_active":"2024-09-13T17:44:50.346093+0000","last_peered":"2024-09-13T17:44:50.346093+0000","last_clean":"2024-09-13T17:44:50.346093+0000","last_became_active":"2024-09-13T17:44:50.341439+0000","last_became_peered":"2024-09-13T17:44:50.341439+0000","last_unstale":"2024-09-13T17:44:50.346093+0000","last_undegraded":"2024-09-13T17:44:50.346093+0000","last_fullsized":"2024-09-13T17:44:50.346093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:29.756925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348557+0000","last_change":"2024-09-13T17:44:50.344536+0000","last_active":"2024-09-13T17:44:50.348557+0000","last_peered":"2024-09-13T17:44:50.348557+0000","last_clean":"2024-09-13T17:44:50.348557+0000","last_became_active":"2024-09-13T17:44:50.339854+0000","last_became_peered":"2024-09-13T17:44:50.339854+0000","last_unstale":"2024-09-13T17:44:50.348557+0000","last_undegraded":"2024-09-13T17:44:50.348557+0000","last_fullsized":"2024-09-13T17:44:50.348557+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:40.121717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335950+0000","last_change":"2024-09-13T17:44:50.332266+0000","last_active":"2024-09-13T17:44:50.335950+0000","last_peered":"2024-09-13T17:44:50.335950+0000","last_clean":"2024-09-13T17:44:50.335950+0000","last_became_active":"2024-09-13T17:44:50.330114+0000","last_became_peered":"2024-09-13T17:44:50.330114+0000","last_unstale":"2024-09-13T17:44:50.335950+0000","last_undegraded":"2024-09-13T17:44:50.335950+0000","last_fullsized":"2024-09-13T17:44:50.335950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:58.004541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301010+0000","last_change":"2024-09-13T17:44:50.332356+0000","last_active":"2024-09-13T17:44:51.301010+0000","last_peered":"2024-09-13T17:44:51.301010+0000","last_clean":"2024-09-13T17:44:51.301010+0000","last_became_active":"2024-09-13T17:44:50.329405+0000","last_became_peered":"2024-09-13T17:44:50.329405+0000","last_unstale":"2024-09-13T17:44:51.301010+0000","last_undegraded":"2024-09-13T17:44:51.301010+0000","last_fullsized":"2024-09-13T17:44:51.301010+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:49.465881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333087+0000","last_change":"2024-09-13T17:44:50.325133+0000","last_active":"2024-09-13T17:44:50.333087+0000","last_peered":"2024-09-13T17:44:50.333087+0000","last_clean":"2024-09-13T17:44:50.333087+0000","last_became_active":"2024-09-13T17:44:50.323005+0000","last_became_peered":"2024-09-13T17:44:50.323005+0000","last_unstale":"2024-09-13T17:44:50.333087+0000","last_undegraded":"2024-09-13T17:44:50.333087+0000","last_fullsized":"2024-09-13T17:44:50.333087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:38.865461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300433+0000","last_change":"2024-09-13T17:44:50.317293+0000","last_active":"2024-09-13T17:44:51.300433+0000","last_peered":"2024-09-13T17:44:51.300433+0000","last_clean":"2024-09-13T17:44:51.300433+0000","last_became_active":"2024-09-13T17:44:50.316974+0000","last_became_peered":"2024-09-13T17:44:50.316974+0000","last_unstale":"2024-09-13T17:44:51.300433+0000","last_undegraded":"2024-09-13T17:44:51.300433+0000","last_fullsized":"2024-09-13T17:44:51.300433+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:50.648279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302094+0000","last_change":"2024-09-13T17:44:50.336056+0000","last_active":"2024-09-13T17:44:51.302094+0000","last_peered":"2024-09-13T17:44:51.302094+0000","last_clean":"2024-09-13T17:44:51.302094+0000","last_became_active":"2024-09-13T17:44:50.334661+0000","last_became_peered":"2024-09-13T17:44:50.334661+0000","last_unstale":"2024-09-13T17:44:51.302094+0000","last_undegraded":"2024-09-13T17:44:51.302094+0000","last_fullsized":"2024-09-13T17:44:51.302094+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:38.778989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348466+0000","last_change":"2024-09-13T17:44:50.342794+0000","last_active":"2024-09-13T17:44:50.348466+0000","last_peered":"2024-09-13T17:44:50.348466+0000","last_clean":"2024-09-13T17:44:50.348466+0000","last_became_active":"2024-09-13T17:44:50.342354+0000","last_became_peered":"2024-09-13T17:44:50.342354+0000","last_unstale":"2024-09-13T17:44:50.348466+0000","last_undegraded":"2024-09-13T17:44:50.348466+0000","last_fullsized":"2024-09-13T17:44:50.348466+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:23.266864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301990+0000","last_change":"2024-09-13T17:44:50.331952+0000","last_active":"2024-09-13T17:44:51.301990+0000","last_peered":"2024-09-13T17:44:51.301990+0000","last_clean":"2024-09-13T17:44:51.301990+0000","last_became_active":"2024-09-13T17:44:50.331369+0000","last_became_peered":"2024-09-13T17:44:50.331369+0000","last_unstale":"2024-09-13T17:44:51.301990+0000","last_undegraded":"2024-09-13T17:44:51.301990+0000","last_fullsized":"2024-09-13T17:44:51.301990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:40.121717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.343000+0000","last_change":"2024-09-13T17:44:50.340821+0000","last_active":"2024-09-13T17:44:50.343000+0000","last_peered":"2024-09-13T17:44:50.343000+0000","last_clean":"2024-09-13T17:44:50.343000+0000","last_became_active":"2024-09-13T17:44:50.340546+0000","last_became_peered":"2024-09-13T17:44:50.340546+0000","last_unstale":"2024-09-13T17:44:50.343000+0000","last_undegraded":"2024-09-13T17:44:50.343000+0000","last_fullsized":"2024-09-13T17:44:50.343000+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:07.216605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.323967+0000","last_change":"2024-09-13T17:44:50.321665+0000","last_active":"2024-09-13T17:44:50.323967+0000","last_peered":"2024-09-13T17:44:50.323967+0000","last_clean":"2024-09-13T17:44:50.323967+0000","last_became_active":"2024-09-13T17:44:50.321257+0000","last_became_peered":"2024-09-13T17:44:50.321257+0000","last_unstale":"2024-09-13T17:44:50.323967+0000","last_undegraded":"2024-09-13T17:44:50.323967+0000","last_fullsized":"2024-09-13T17:44:50.323967+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:21.931915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301010+0000","last_change":"2024-09-13T17:44:50.332522+0000","last_active":"2024-09-13T17:44:51.301010+0000","last_peered":"2024-09-13T17:44:51.301010+0000","last_clean":"2024-09-13T17:44:51.301010+0000","last_became_active":"2024-09-13T17:44:50.329324+0000","last_became_peered":"2024-09-13T17:44:50.329324+0000","last_unstale":"2024-09-13T17:44:51.301010+0000","last_undegraded":"2024-09-13T17:44:51.301010+0000","last_fullsized":"2024-09-13T17:44:51.301010+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:55.586004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324542+0000","last_change":"2024-09-13T17:44:50.323392+0000","last_active":"2024-09-13T17:44:50.324542+0000","last_peered":"2024-09-13T17:44:50.324542+0000","last_clean":"2024-09-13T17:44:50.324542+0000","last_became_active":"2024-09-13T17:44:50.323154+0000","last_became_peered":"2024-09-13T17:44:50.323154+0000","last_unstale":"2024-09-13T17:44:50.324542+0000","last_undegraded":"2024-09-13T17:44:50.324542+0000","last_fullsized":"2024-09-13T17:44:50.324542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:55.794634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346082+0000","last_change":"2024-09-13T17:44:50.341383+0000","last_active":"2024-09-13T17:44:50.346082+0000","last_peered":"2024-09-13T17:44:50.346082+0000","last_clean":"2024-09-13T17:44:50.346082+0000","last_became_active":"2024-09-13T17:44:50.340966+0000","last_became_peered":"2024-09-13T17:44:50.340966+0000","last_unstale":"2024-09-13T17:44:50.346082+0000","last_undegraded":"2024-09-13T17:44:50.346082+0000","last_fullsized":"2024-09-13T17:44:50.346082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:47.271487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326049+0000","last_change":"2024-09-13T17:44:50.320797+0000","last_active":"2024-09-13T17:44:50.326049+0000","last_peered":"2024-09-13T17:44:50.326049+0000","last_clean":"2024-09-13T17:44:50.326049+0000","last_became_active":"2024-09-13T17:44:50.320545+0000","last_became_peered":"2024-09-13T17:44:50.320545+0000","last_unstale":"2024-09-13T17:44:50.326049+0000","last_undegraded":"2024-09-13T17:44:50.326049+0000","last_fullsized":"2024-09-13T17:44:50.326049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:10:59.117315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346177+0000","last_change":"2024-09-13T17:44:50.342612+0000","last_active":"2024-09-13T17:44:50.346177+0000","last_peered":"2024-09-13T17:44:50.346177+0000","last_clean":"2024-09-13T17:44:50.346177+0000","last_became_active":"2024-09-13T17:44:50.342168+0000","last_became_peered":"2024-09-13T17:44:50.342168+0000","last_unstale":"2024-09-13T17:44:50.346177+0000","last_undegraded":"2024-09-13T17:44:50.346177+0000","last_fullsized":"2024-09-13T17:44:50.346177+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:57:37.107184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335294+0000","last_change":"2024-09-13T17:44:50.328811+0000","last_active":"2024-09-13T17:44:50.335294+0000","last_peered":"2024-09-13T17:44:50.335294+0000","last_clean":"2024-09-13T17:44:50.335294+0000","last_became_active":"2024-09-13T17:44:50.328561+0000","last_became_peered":"2024-09-13T17:44:50.328561+0000","last_unstale":"2024-09-13T17:44:50.335294+0000","last_undegraded":"2024-09-13T17:44:50.335294+0000","last_fullsized":"2024-09-13T17:44:50.335294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:36.490472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.343020+0000","last_change":"2024-09-13T17:44:50.340883+0000","last_active":"2024-09-13T17:44:50.343020+0000","last_peered":"2024-09-13T17:44:50.343020+0000","last_clean":"2024-09-13T17:44:50.343020+0000","last_became_active":"2024-09-13T17:44:50.340580+0000","last_became_peered":"2024-09-13T17:44:50.340580+0000","last_unstale":"2024-09-13T17:44:50.343020+0000","last_undegraded":"2024-09-13T17:44:50.343020+0000","last_fullsized":"2024-09-13T17:44:50.343020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:07.932090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333514+0000","last_change":"2024-09-13T17:44:50.326057+0000","last_active":"2024-09-13T17:44:50.333514+0000","last_peered":"2024-09-13T17:44:50.333514+0000","last_clean":"2024-09-13T17:44:50.333514+0000","last_became_active":"2024-09-13T17:44:50.320825+0000","last_became_peered":"2024-09-13T17:44:50.320825+0000","last_unstale":"2024-09-13T17:44:50.333514+0000","last_undegraded":"2024-09-13T17:44:50.333514+0000","last_fullsized":"2024-09-13T17:44:50.333514+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:49.465881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324447+0000","last_change":"2024-09-13T17:44:50.321619+0000","last_active":"2024-09-13T17:44:50.324447+0000","last_peered":"2024-09-13T17:44:50.324447+0000","last_clean":"2024-09-13T17:44:50.324447+0000","last_became_active":"2024-09-13T17:44:50.321300+0000","last_became_peered":"2024-09-13T17:44:50.321300+0000","last_unstale":"2024-09-13T17:44:50.324447+0000","last_undegraded":"2024-09-13T17:44:50.324447+0000","last_fullsized":"2024-09-13T17:44:50.324447+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:50.641774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.325430+0000","last_change":"2024-09-13T17:44:50.322775+0000","last_active":"2024-09-13T17:44:50.325430+0000","last_peered":"2024-09-13T17:44:50.325430+0000","last_clean":"2024-09-13T17:44:50.325430+0000","last_became_active":"2024-09-13T17:44:50.322517+0000","last_became_peered":"2024-09-13T17:44:50.322517+0000","last_unstale":"2024-09-13T17:44:50.325430+0000","last_undegraded":"2024-09-13T17:44:50.325430+0000","last_fullsized":"2024-09-13T17:44:50.325430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:44.502496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324572+0000","last_change":"2024-09-13T17:44:50.322670+0000","last_active":"2024-09-13T17:44:50.324572+0000","last_peered":"2024-09-13T17:44:50.324572+0000","last_clean":"2024-09-13T17:44:50.324572+0000","last_became_active":"2024-09-13T17:44:50.322159+0000","last_became_peered":"2024-09-13T17:44:50.322159+0000","last_unstale":"2024-09-13T17:44:50.324572+0000","last_undegraded":"2024-09-13T17:44:50.324572+0000","last_fullsized":"2024-09-13T17:44:50.324572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:56.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302861+0000","last_change":"2024-09-13T17:44:50.319795+0000","last_active":"2024-09-13T17:44:51.302861+0000","last_peered":"2024-09-13T17:44:51.302861+0000","last_clean":"2024-09-13T17:44:51.302861+0000","last_became_active":"2024-09-13T17:44:50.319553+0000","last_became_peered":"2024-09-13T17:44:50.319553+0000","last_unstale":"2024-09-13T17:44:51.302861+0000","last_undegraded":"2024-09-13T17:44:51.302861+0000","last_fullsized":"2024-09-13T17:44:51.302861+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:44.728477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1f","version":"55'4","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302480+0000","last_change":"2024-09-13T17:44:50.010193+0000","last_active":"2024-09-13T17:44:51.302480+0000","last_peered":"2024-09-13T17:44:51.302480+0000","last_clean":"2024-09-13T17:44:51.302480+0000","last_became_active":"2024-09-13T17:44:50.009934+0000","last_became_peered":"2024-09-13T17:44:50.009934+0000","last_unstale":"2024-09-13T17:44:51.302480+0000","last_undegraded":"2024-09-13T17:44:51.302480+0000","last_fullsized":"2024-09-13T17:44:51.302480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:07:21.419980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302090+0000","last_change":"2024-09-13T17:44:50.336829+0000","last_active":"2024-09-13T17:44:51.302090+0000","last_peered":"2024-09-13T17:44:51.302090+0000","last_clean":"2024-09-13T17:44:51.302090+0000","last_became_active":"2024-09-13T17:44:50.336470+0000","last_became_peered":"2024-09-13T17:44:50.336470+0000","last_unstale":"2024-09-13T17:44:51.302090+0000","last_undegraded":"2024-09-13T17:44:51.302090+0000","last_fullsized":"2024-09-13T17:44:51.302090+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:30.343357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.340244+0000","last_change":"2024-09-13T17:44:50.019324+0000","last_active":"2024-09-13T17:44:50.340244+0000","last_peered":"2024-09-13T17:44:50.340244+0000","last_clean":"2024-09-13T17:44:50.340244+0000","last_became_active":"2024-09-13T17:44:50.019017+0000","last_became_peered":"2024-09-13T17:44:50.019017+0000","last_unstale":"2024-09-13T17:44:50.340244+0000","last_undegraded":"2024-09-13T17:44:50.340244+0000","last_fullsized":"2024-09-13T17:44:50.340244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:41.923832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301742+0000","last_change":"2024-09-13T17:44:50.322892+0000","last_active":"2024-09-13T17:44:51.301742+0000","last_peered":"2024-09-13T17:44:51.301742+0000","last_clean":"2024-09-13T17:44:51.301742+0000","last_became_active":"2024-09-13T17:44:50.318045+0000","last_became_peered":"2024-09-13T17:44:50.318045+0000","last_unstale":"2024-09-13T17:44:51.301742+0000","last_undegraded":"2024-09-13T17:44:51.301742+0000","last_fullsized":"2024-09-13T17:44:51.301742+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:09.358687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"55'2","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302607+0000","last_change":"2024-09-13T17:44:50.010620+0000","last_active":"2024-09-13T17:44:51.302607+0000","last_peered":"2024-09-13T17:44:51.302607+0000","last_clean":"2024-09-13T17:44:51.302607+0000","last_became_active":"2024-09-13T17:44:50.009775+0000","last_became_peered":"2024-09-13T17:44:50.009775+0000","last_unstale":"2024-09-13T17:44:51.302607+0000","last_undegraded":"2024-09-13T17:44:51.302607+0000","last_fullsized":"2024-09-13T17:44:51.302607+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:45.360880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333653+0000","last_change":"2024-09-13T17:44:50.326684+0000","last_active":"2024-09-13T17:44:50.333653+0000","last_peered":"2024-09-13T17:44:50.333653+0000","last_clean":"2024-09-13T17:44:50.333653+0000","last_became_active":"2024-09-13T17:44:50.326364+0000","last_became_peered":"2024-09-13T17:44:50.326364+0000","last_unstale":"2024-09-13T17:44:50.333653+0000","last_undegraded":"2024-09-13T17:44:50.333653+0000","last_fullsized":"2024-09-13T17:44:50.333653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:39.872446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"55'2","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301913+0000","last_change":"2024-09-13T17:44:50.008813+0000","last_active":"2024-09-13T17:44:51.301913+0000","last_peered":"2024-09-13T17:44:51.301913+0000","last_clean":"2024-09-13T17:44:51.301913+0000","last_became_active":"2024-09-13T17:44:50.008496+0000","last_became_peered":"2024-09-13T17:44:50.008496+0000","last_unstale":"2024-09-13T17:44:51.301913+0000","last_undegraded":"2024-09-13T17:44:51.301913+0000","last_fullsized":"2024-09-13T17:44:51.301913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:50.066142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301122+0000","last_change":"2024-09-13T17:44:50.328909+0000","last_active":"2024-09-13T17:44:51.301122+0000","last_peered":"2024-09-13T17:44:51.301122+0000","last_clean":"2024-09-13T17:44:51.301122+0000","last_became_active":"2024-09-13T17:44:50.320247+0000","last_became_peered":"2024-09-13T17:44:50.320247+0000","last_unstale":"2024-09-13T17:44:51.301122+0000","last_undegraded":"2024-09-13T17:44:51.301122+0000","last_fullsized":"2024-09-13T17:44:51.301122+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:03.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"55'1","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301691+0000","last_change":"2024-09-13T17:44:50.009366+0000","last_active":"2024-09-13T17:44:51.301691+0000","last_peered":"2024-09-13T17:44:51.301691+0000","last_clean":"2024-09-13T17:44:51.301691+0000","last_became_active":"2024-09-13T17:44:50.009055+0000","last_became_peered":"2024-09-13T17:44:50.009055+0000","last_unstale":"2024-09-13T17:44:51.301691+0000","last_undegraded":"2024-09-13T17:44:51.301691+0000","last_fullsized":"2024-09-13T17:44:51.301691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:15.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301430+0000","last_change":"2024-09-13T17:44:50.336546+0000","last_active":"2024-09-13T17:44:51.301430+0000","last_peered":"2024-09-13T17:44:51.301430+0000","last_clean":"2024-09-13T17:44:51.301430+0000","last_became_active":"2024-09-13T17:44:50.336264+0000","last_became_peered":"2024-09-13T17:44:50.336264+0000","last_unstale":"2024-09-13T17:44:51.301430+0000","last_undegraded":"2024-09-13T17:44:51.301430+0000","last_fullsized":"2024-09-13T17:44:51.301430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:31.116346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.347701+0000","last_change":"2024-09-13T17:44:50.016616+0000","last_active":"2024-09-13T17:44:50.347701+0000","last_peered":"2024-09-13T17:44:50.347701+0000","last_clean":"2024-09-13T17:44:50.347701+0000","last_became_active":"2024-09-13T17:44:50.016359+0000","last_became_peered":"2024-09-13T17:44:50.016359+0000","last_unstale":"2024-09-13T17:44:50.347701+0000","last_undegraded":"2024-09-13T17:44:50.347701+0000","last_fullsized":"2024-09-13T17:44:50.347701+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:50.066142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300491+0000","last_change":"2024-09-13T17:44:50.332439+0000","last_active":"2024-09-13T17:44:51.300491+0000","last_peered":"2024-09-13T17:44:51.300491+0000","last_clean":"2024-09-13T17:44:51.300491+0000","last_became_active":"2024-09-13T17:44:50.332037+0000","last_became_peered":"2024-09-13T17:44:50.332037+0000","last_unstale":"2024-09-13T17:44:51.300491+0000","last_undegraded":"2024-09-13T17:44:51.300491+0000","last_fullsized":"2024-09-13T17:44:51.300491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:11.433421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.311633+0000","last_change":"2024-09-13T17:44:50.013828+0000","last_active":"2024-09-13T17:44:50.311633+0000","last_peered":"2024-09-13T17:44:50.311633+0000","last_clean":"2024-09-13T17:44:50.311633+0000","last_became_active":"2024-09-13T17:44:50.013573+0000","last_became_peered":"2024-09-13T17:44:50.013573+0000","last_unstale":"2024-09-13T17:44:50.311633+0000","last_undegraded":"2024-09-13T17:44:50.311633+0000","last_fullsized":"2024-09-13T17:44:50.311633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:42.012204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335868+0000","last_change":"2024-09-13T17:44:50.331174+0000","last_active":"2024-09-13T17:44:50.335868+0000","last_peered":"2024-09-13T17:44:50.335868+0000","last_clean":"2024-09-13T17:44:50.335868+0000","last_became_active":"2024-09-13T17:44:50.330459+0000","last_became_peered":"2024-09-13T17:44:50.330459+0000","last_unstale":"2024-09-13T17:44:50.335868+0000","last_undegraded":"2024-09-13T17:44:50.335868+0000","last_fullsized":"2024-09-13T17:44:50.335868+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:17.418015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300568+0000","last_change":"2024-09-13T17:44:50.010224+0000","last_active":"2024-09-13T17:44:51.300568+0000","last_peered":"2024-09-13T17:44:51.300568+0000","last_clean":"2024-09-13T17:44:51.300568+0000","last_became_active":"2024-09-13T17:44:50.008628+0000","last_became_peered":"2024-09-13T17:44:50.008628+0000","last_unstale":"2024-09-13T17:44:51.300568+0000","last_undegraded":"2024-09-13T17:44:51.300568+0000","last_fullsized":"2024-09-13T17:44:51.300568+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:21.595209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324628+0000","last_change":"2024-09-13T17:44:50.322139+0000","last_active":"2024-09-13T17:44:50.324628+0000","last_peered":"2024-09-13T17:44:50.324628+0000","last_clean":"2024-09-13T17:44:50.324628+0000","last_became_active":"2024-09-13T17:44:50.321994+0000","last_became_peered":"2024-09-13T17:44:50.321994+0000","last_unstale":"2024-09-13T17:44:50.324628+0000","last_undegraded":"2024-09-13T17:44:50.324628+0000","last_fullsized":"2024-09-13T17:44:50.324628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:40.580340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305889+0000","last_change":"2024-09-13T17:44:49.947282+0000","last_active":"2024-09-13T17:44:50.305889+0000","last_peered":"2024-09-13T17:44:50.305889+0000","last_clean":"2024-09-13T17:44:50.305889+0000","last_became_active":"2024-09-13T17:44:49.947009+0000","last_became_peered":"2024-09-13T17:44:49.947009+0000","last_unstale":"2024-09-13T17:44:50.305889+0000","last_undegraded":"2024-09-13T17:44:50.305889+0000","last_fullsized":"2024-09-13T17:44:50.305889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:31.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335955+0000","last_change":"2024-09-13T17:44:50.330889+0000","last_active":"2024-09-13T17:44:50.335955+0000","last_peered":"2024-09-13T17:44:50.335955+0000","last_clean":"2024-09-13T17:44:50.335955+0000","last_became_active":"2024-09-13T17:44:50.330660+0000","last_became_peered":"2024-09-13T17:44:50.330660+0000","last_unstale":"2024-09-13T17:44:50.335955+0000","last_undegraded":"2024-09-13T17:44:50.335955+0000","last_fullsized":"2024-09-13T17:44:50.335955+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:47.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.306472+0000","last_change":"2024-09-13T17:44:50.019871+0000","last_active":"2024-09-13T17:44:50.306472+0000","last_peered":"2024-09-13T17:44:50.306472+0000","last_clean":"2024-09-13T17:44:50.306472+0000","last_became_active":"2024-09-13T17:44:50.019125+0000","last_became_peered":"2024-09-13T17:44:50.019125+0000","last_unstale":"2024-09-13T17:44:50.306472+0000","last_undegraded":"2024-09-13T17:44:50.306472+0000","last_fullsized":"2024-09-13T17:44:50.306472+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:58.749723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335492+0000","last_change":"2024-09-13T17:44:50.329956+0000","last_active":"2024-09-13T17:44:50.335492+0000","last_peered":"2024-09-13T17:44:50.335492+0000","last_clean":"2024-09-13T17:44:50.335492+0000","last_became_active":"2024-09-13T17:44:50.328065+0000","last_became_peered":"2024-09-13T17:44:50.328065+0000","last_unstale":"2024-09-13T17:44:50.335492+0000","last_undegraded":"2024-09-13T17:44:50.335492+0000","last_fullsized":"2024-09-13T17:44:50.335492+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:34.070976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"55'3","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.341223+0000","last_change":"2024-09-13T17:44:50.018014+0000","last_active":"2024-09-13T17:44:50.341223+0000","last_peered":"2024-09-13T17:44:50.341223+0000","last_clean":"2024-09-13T17:44:50.341223+0000","last_became_active":"2024-09-13T17:44:50.017758+0000","last_became_peered":"2024-09-13T17:44:50.017758+0000","last_unstale":"2024-09-13T17:44:50.341223+0000","last_undegraded":"2024-09-13T17:44:50.341223+0000","last_fullsized":"2024-09-13T17:44:50.341223+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:59.256249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302117+0000","last_change":"2024-09-13T17:44:50.331791+0000","last_active":"2024-09-13T17:44:51.302117+0000","last_peered":"2024-09-13T17:44:51.302117+0000","last_clean":"2024-09-13T17:44:51.302117+0000","last_became_active":"2024-09-13T17:44:50.328573+0000","last_became_peered":"2024-09-13T17:44:50.328573+0000","last_unstale":"2024-09-13T17:44:51.302117+0000","last_undegraded":"2024-09-13T17:44:51.302117+0000","last_fullsized":"2024-09-13T17:44:51.302117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:16.762863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"55'6","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349747+0000","last_change":"2024-09-13T17:44:50.014447+0000","last_active":"2024-09-13T17:44:50.349747+0000","last_peered":"2024-09-13T17:44:50.349747+0000","last_clean":"2024-09-13T17:44:50.349747+0000","last_became_active":"2024-09-13T17:44:50.014096+0000","last_became_peered":"2024-09-13T17:44:50.014096+0000","last_unstale":"2024-09-13T17:44:50.349747+0000","last_undegraded":"2024-09-13T17:44:50.349747+0000","last_fullsized":"2024-09-13T17:44:50.349747+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:50:55.839610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301214+0000","last_change":"2024-09-13T17:44:50.329075+0000","last_active":"2024-09-13T17:44:51.301214+0000","last_peered":"2024-09-13T17:44:51.301214+0000","last_clean":"2024-09-13T17:44:51.301214+0000","last_became_active":"2024-09-13T17:44:50.320531+0000","last_became_peered":"2024-09-13T17:44:50.320531+0000","last_unstale":"2024-09-13T17:44:51.301214+0000","last_undegraded":"2024-09-13T17:44:51.301214+0000","last_fullsized":"2024-09-13T17:44:51.301214+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:37.332944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.347530+0000","last_change":"2024-09-13T17:44:50.013584+0000","last_active":"2024-09-13T17:44:50.347530+0000","last_peered":"2024-09-13T17:44:50.347530+0000","last_clean":"2024-09-13T17:44:50.347530+0000","last_became_active":"2024-09-13T17:44:50.013319+0000","last_became_peered":"2024-09-13T17:44:50.013319+0000","last_unstale":"2024-09-13T17:44:50.347530+0000","last_undegraded":"2024-09-13T17:44:50.347530+0000","last_fullsized":"2024-09-13T17:44:50.347530+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:43:52.006225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326198+0000","last_change":"2024-09-13T17:44:50.324064+0000","last_active":"2024-09-13T17:44:50.326198+0000","last_peered":"2024-09-13T17:44:50.326198+0000","last_clean":"2024-09-13T17:44:50.326198+0000","last_became_active":"2024-09-13T17:44:50.323879+0000","last_became_peered":"2024-09-13T17:44:50.323879+0000","last_unstale":"2024-09-13T17:44:50.326198+0000","last_undegraded":"2024-09-13T17:44:50.326198+0000","last_fullsized":"2024-09-13T17:44:50.326198+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:16.762863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312470+0000","last_change":"2024-09-13T17:44:50.016629+0000","last_active":"2024-09-13T17:44:50.312470+0000","last_peered":"2024-09-13T17:44:50.312470+0000","last_clean":"2024-09-13T17:44:50.312470+0000","last_became_active":"2024-09-13T17:44:50.016072+0000","last_became_peered":"2024-09-13T17:44:50.016072+0000","last_unstale":"2024-09-13T17:44:50.312470+0000","last_undegraded":"2024-09-13T17:44:50.312470+0000","last_fullsized":"2024-09-13T17:44:50.312470+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:07:21.419980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346069+0000","last_change":"2024-09-13T17:44:50.341042+0000","last_active":"2024-09-13T17:44:50.346069+0000","last_peered":"2024-09-13T17:44:50.346069+0000","last_clean":"2024-09-13T17:44:50.346069+0000","last_became_active":"2024-09-13T17:44:50.340617+0000","last_became_peered":"2024-09-13T17:44:50.340617+0000","last_unstale":"2024-09-13T17:44:50.346069+0000","last_undegraded":"2024-09-13T17:44:50.346069+0000","last_fullsized":"2024-09-13T17:44:50.346069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:08.078983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.311724+0000","last_change":"2024-09-13T17:44:49.946367+0000","last_active":"2024-09-13T17:44:50.311724+0000","last_peered":"2024-09-13T17:44:50.311724+0000","last_clean":"2024-09-13T17:44:50.311724+0000","last_became_active":"2024-09-13T17:44:49.943792+0000","last_became_peered":"2024-09-13T17:44:49.943792+0000","last_unstale":"2024-09-13T17:44:50.311724+0000","last_undegraded":"2024-09-13T17:44:50.311724+0000","last_fullsized":"2024-09-13T17:44:50.311724+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:52.214852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301144+0000","last_change":"2024-09-13T17:44:50.323169+0000","last_active":"2024-09-13T17:44:51.301144+0000","last_peered":"2024-09-13T17:44:51.301144+0000","last_clean":"2024-09-13T17:44:51.301144+0000","last_became_active":"2024-09-13T17:44:50.322913+0000","last_became_peered":"2024-09-13T17:44:50.322913+0000","last_unstale":"2024-09-13T17:44:51.301144+0000","last_undegraded":"2024-09-13T17:44:51.301144+0000","last_fullsized":"2024-09-13T17:44:51.301144+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:02.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"55'8","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348180+0000","last_change":"2024-09-13T17:44:50.018798+0000","last_active":"2024-09-13T17:44:50.348180+0000","last_peered":"2024-09-13T17:44:50.348180+0000","last_clean":"2024-09-13T17:44:50.348180+0000","last_became_active":"2024-09-13T17:44:50.018492+0000","last_became_peered":"2024-09-13T17:44:50.018492+0000","last_unstale":"2024-09-13T17:44:50.348180+0000","last_undegraded":"2024-09-13T17:44:50.348180+0000","last_fullsized":"2024-09-13T17:44:50.348180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:15.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335988+0000","last_change":"2024-09-13T17:44:50.333401+0000","last_active":"2024-09-13T17:44:50.335988+0000","last_peered":"2024-09-13T17:44:50.335988+0000","last_clean":"2024-09-13T17:44:50.335988+0000","last_became_active":"2024-09-13T17:44:50.333129+0000","last_became_peered":"2024-09-13T17:44:50.333129+0000","last_unstale":"2024-09-13T17:44:50.335988+0000","last_undegraded":"2024-09-13T17:44:50.335988+0000","last_fullsized":"2024-09-13T17:44:50.335988+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:04.136915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.345041+0000","last_change":"2024-09-13T17:44:50.016270+0000","last_active":"2024-09-13T17:44:50.345041+0000","last_peered":"2024-09-13T17:44:50.345041+0000","last_clean":"2024-09-13T17:44:50.345041+0000","last_became_active":"2024-09-13T17:44:50.016025+0000","last_became_peered":"2024-09-13T17:44:50.016025+0000","last_unstale":"2024-09-13T17:44:50.345041+0000","last_undegraded":"2024-09-13T17:44:50.345041+0000","last_fullsized":"2024-09-13T17:44:50.345041+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:52.332294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335939+0000","last_change":"2024-09-13T17:44:50.330830+0000","last_active":"2024-09-13T17:44:50.335939+0000","last_peered":"2024-09-13T17:44:50.335939+0000","last_clean":"2024-09-13T17:44:50.335939+0000","last_became_active":"2024-09-13T17:44:50.330588+0000","last_became_peered":"2024-09-13T17:44:50.330588+0000","last_unstale":"2024-09-13T17:44:50.335939+0000","last_undegraded":"2024-09-13T17:44:50.335939+0000","last_fullsized":"2024-09-13T17:44:50.335939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:37.132544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.328872+0000","last_change":"2024-09-13T17:44:49.947334+0000","last_active":"2024-09-13T17:44:50.328872+0000","last_peered":"2024-09-13T17:44:50.328872+0000","last_clean":"2024-09-13T17:44:50.328872+0000","last_became_active":"2024-09-13T17:44:49.946893+0000","last_became_peered":"2024-09-13T17:44:49.946893+0000","last_unstale":"2024-09-13T17:44:50.328872+0000","last_undegraded":"2024-09-13T17:44:50.328872+0000","last_fullsized":"2024-09-13T17:44:50.328872+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:49.638098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349264+0000","last_change":"2024-09-13T17:44:50.344731+0000","last_active":"2024-09-13T17:44:50.349264+0000","last_peered":"2024-09-13T17:44:50.349264+0000","last_clean":"2024-09-13T17:44:50.349264+0000","last_became_active":"2024-09-13T17:44:50.340212+0000","last_became_peered":"2024-09-13T17:44:50.340212+0000","last_unstale":"2024-09-13T17:44:50.349264+0000","last_undegraded":"2024-09-13T17:44:50.349264+0000","last_fullsized":"2024-09-13T17:44:50.349264+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:02.534122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.341240+0000","last_change":"2024-09-13T17:44:50.017970+0000","last_active":"2024-09-13T17:44:50.341240+0000","last_peered":"2024-09-13T17:44:50.341240+0000","last_clean":"2024-09-13T17:44:50.341240+0000","last_became_active":"2024-09-13T17:44:50.017421+0000","last_became_peered":"2024-09-13T17:44:50.017421+0000","last_unstale":"2024-09-13T17:44:50.341240+0000","last_undegraded":"2024-09-13T17:44:50.341240+0000","last_fullsized":"2024-09-13T17:44:50.341240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:58:07.964647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302545+0000","last_change":"2024-09-13T17:44:50.331615+0000","last_active":"2024-09-13T17:44:51.302545+0000","last_peered":"2024-09-13T17:44:51.302545+0000","last_clean":"2024-09-13T17:44:51.302545+0000","last_became_active":"2024-09-13T17:44:50.328279+0000","last_became_peered":"2024-09-13T17:44:50.328279+0000","last_unstale":"2024-09-13T17:44:51.302545+0000","last_undegraded":"2024-09-13T17:44:51.302545+0000","last_fullsized":"2024-09-13T17:44:51.302545+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:39.872446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301825+0000","last_change":"2024-09-13T17:44:50.008833+0000","last_active":"2024-09-13T17:44:51.301825+0000","last_peered":"2024-09-13T17:44:51.301825+0000","last_clean":"2024-09-13T17:44:51.301825+0000","last_became_active":"2024-09-13T17:44:50.008509+0000","last_became_peered":"2024-09-13T17:44:50.008509+0000","last_unstale":"2024-09-13T17:44:51.301825+0000","last_undegraded":"2024-09-13T17:44:51.301825+0000","last_fullsized":"2024-09-13T17:44:51.301825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:33.042083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301688+0000","last_change":"2024-09-13T17:44:50.336084+0000","last_active":"2024-09-13T17:44:51.301688+0000","last_peered":"2024-09-13T17:44:51.301688+0000","last_clean":"2024-09-13T17:44:51.301688+0000","last_became_active":"2024-09-13T17:44:50.334621+0000","last_became_peered":"2024-09-13T17:44:50.334621+0000","last_unstale":"2024-09-13T17:44:51.301688+0000","last_undegraded":"2024-09-13T17:44:51.301688+0000","last_fullsized":"2024-09-13T17:44:51.301688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:02.534122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301280+0000","last_change":"2024-09-13T17:44:50.012564+0000","last_active":"2024-09-13T17:44:51.301280+0000","last_peered":"2024-09-13T17:44:51.301280+0000","last_clean":"2024-09-13T17:44:51.301280+0000","last_became_active":"2024-09-13T17:44:50.011951+0000","last_became_peered":"2024-09-13T17:44:50.011951+0000","last_unstale":"2024-09-13T17:44:51.301280+0000","last_undegraded":"2024-09-13T17:44:51.301280+0000","last_fullsized":"2024-09-13T17:44:51.301280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:52.214852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300528+0000","last_change":"2024-09-13T17:44:50.332068+0000","last_active":"2024-09-13T17:44:51.300528+0000","last_peered":"2024-09-13T17:44:51.300528+0000","last_clean":"2024-09-13T17:44:51.300528+0000","last_became_active":"2024-09-13T17:44:50.331457+0000","last_became_peered":"2024-09-13T17:44:50.331457+0000","last_unstale":"2024-09-13T17:44:51.300528+0000","last_undegraded":"2024-09-13T17:44:51.300528+0000","last_fullsized":"2024-09-13T17:44:51.300528+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:33.970616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305135+0000","last_change":"2024-09-13T17:44:50.019696+0000","last_active":"2024-09-13T17:44:50.305135+0000","last_peered":"2024-09-13T17:44:50.305135+0000","last_clean":"2024-09-13T17:44:50.305135+0000","last_became_active":"2024-09-13T17:44:50.019417+0000","last_became_peered":"2024-09-13T17:44:50.019417+0000","last_unstale":"2024-09-13T17:44:50.305135+0000","last_undegraded":"2024-09-13T17:44:50.305135+0000","last_fullsized":"2024-09-13T17:44:50.305135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:51.953778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301305+0000","last_change":"2024-09-13T17:44:50.327682+0000","last_active":"2024-09-13T17:44:51.301305+0000","last_peered":"2024-09-13T17:44:51.301305+0000","last_clean":"2024-09-13T17:44:51.301305+0000","last_became_active":"2024-09-13T17:44:50.327422+0000","last_became_peered":"2024-09-13T17:44:50.327422+0000","last_unstale":"2024-09-13T17:44:51.301305+0000","last_undegraded":"2024-09-13T17:44:51.301305+0000","last_fullsized":"2024-09-13T17:44:51.301305+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:10:59.117315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.304168+0000","last_change":"2024-09-13T17:44:50.017876+0000","last_active":"2024-09-13T17:44:50.304168+0000","last_peered":"2024-09-13T17:44:50.304168+0000","last_clean":"2024-09-13T17:44:50.304168+0000","last_became_active":"2024-09-13T17:44:50.017576+0000","last_became_peered":"2024-09-13T17:44:50.017576+0000","last_unstale":"2024-09-13T17:44:50.304168+0000","last_undegraded":"2024-09-13T17:44:50.304168+0000","last_fullsized":"2024-09-13T17:44:50.304168+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:11:32.395758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324992+0000","last_change":"2024-09-13T17:44:50.322119+0000","last_active":"2024-09-13T17:44:50.324992+0000","last_peered":"2024-09-13T17:44:50.324992+0000","last_clean":"2024-09-13T17:44:50.324992+0000","last_became_active":"2024-09-13T17:44:50.321856+0000","last_became_peered":"2024-09-13T17:44:50.321856+0000","last_unstale":"2024-09-13T17:44:50.324992+0000","last_undegraded":"2024-09-13T17:44:50.324992+0000","last_fullsized":"2024-09-13T17:44:50.324992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:57.254892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335849+0000","last_change":"2024-09-13T17:44:49.945926+0000","last_active":"2024-09-13T17:44:50.335849+0000","last_peered":"2024-09-13T17:44:50.335849+0000","last_clean":"2024-09-13T17:44:50.335849+0000","last_became_active":"2024-09-13T17:44:49.945490+0000","last_became_peered":"2024-09-13T17:44:49.945490+0000","last_unstale":"2024-09-13T17:44:50.335849+0000","last_undegraded":"2024-09-13T17:44:50.335849+0000","last_fullsized":"2024-09-13T17:44:50.335849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:50.208962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326028+0000","last_change":"2024-09-13T17:44:50.320331+0000","last_active":"2024-09-13T17:44:50.326028+0000","last_peered":"2024-09-13T17:44:50.326028+0000","last_clean":"2024-09-13T17:44:50.326028+0000","last_became_active":"2024-09-13T17:44:50.320061+0000","last_became_peered":"2024-09-13T17:44:50.320061+0000","last_unstale":"2024-09-13T17:44:50.326028+0000","last_undegraded":"2024-09-13T17:44:50.326028+0000","last_fullsized":"2024-09-13T17:44:50.326028+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:51.647187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312379+0000","last_change":"2024-09-13T17:44:49.944249+0000","last_active":"2024-09-13T17:44:50.312379+0000","last_peered":"2024-09-13T17:44:50.312379+0000","last_clean":"2024-09-13T17:44:50.312379+0000","last_became_active":"2024-09-13T17:44:49.943968+0000","last_became_peered":"2024-09-13T17:44:49.943968+0000","last_unstale":"2024-09-13T17:44:50.312379+0000","last_undegraded":"2024-09-13T17:44:50.312379+0000","last_fullsized":"2024-09-13T17:44:50.312379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:37.117449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324481+0000","last_change":"2024-09-13T17:44:50.322659+0000","last_active":"2024-09-13T17:44:50.324481+0000","last_peered":"2024-09-13T17:44:50.324481+0000","last_clean":"2024-09-13T17:44:50.324481+0000","last_became_active":"2024-09-13T17:44:50.322107+0000","last_became_peered":"2024-09-13T17:44:50.322107+0000","last_unstale":"2024-09-13T17:44:50.324481+0000","last_undegraded":"2024-09-13T17:44:50.324481+0000","last_fullsized":"2024-09-13T17:44:50.324481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:35.827537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302656+0000","last_change":"2024-09-13T17:44:50.012895+0000","last_active":"2024-09-13T17:44:51.302656+0000","last_peered":"2024-09-13T17:44:51.302656+0000","last_clean":"2024-09-13T17:44:51.302656+0000","last_became_active":"2024-09-13T17:44:50.012644+0000","last_became_peered":"2024-09-13T17:44:50.012644+0000","last_unstale":"2024-09-13T17:44:51.302656+0000","last_undegraded":"2024-09-13T17:44:51.302656+0000","last_fullsized":"2024-09-13T17:44:51.302656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:37.117449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346350+0000","last_change":"2024-09-13T17:44:50.342649+0000","last_active":"2024-09-13T17:44:50.346350+0000","last_peered":"2024-09-13T17:44:50.346350+0000","last_clean":"2024-09-13T17:44:50.346350+0000","last_became_active":"2024-09-13T17:44:50.339607+0000","last_became_peered":"2024-09-13T17:44:50.339607+0000","last_unstale":"2024-09-13T17:44:50.346350+0000","last_undegraded":"2024-09-13T17:44:50.346350+0000","last_fullsized":"2024-09-13T17:44:50.346350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:30.343357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305771+0000","last_change":"2024-09-13T17:44:49.945906+0000","last_active":"2024-09-13T17:44:50.305771+0000","last_peered":"2024-09-13T17:44:50.305771+0000","last_clean":"2024-09-13T17:44:50.305771+0000","last_became_active":"2024-09-13T17:44:49.945059+0000","last_became_peered":"2024-09-13T17:44:49.945059+0000","last_unstale":"2024-09-13T17:44:50.305771+0000","last_undegraded":"2024-09-13T17:44:50.305771+0000","last_fullsized":"2024-09-13T17:44:50.305771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:34.030988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301461+0000","last_change":"2024-09-13T17:44:50.331088+0000","last_active":"2024-09-13T17:44:51.301461+0000","last_peered":"2024-09-13T17:44:51.301461+0000","last_clean":"2024-09-13T17:44:51.301461+0000","last_became_active":"2024-09-13T17:44:50.329057+0000","last_became_peered":"2024-09-13T17:44:50.329057+0000","last_unstale":"2024-09-13T17:44:51.301461+0000","last_undegraded":"2024-09-13T17:44:51.301461+0000","last_fullsized":"2024-09-13T17:44:51.301461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:49.880699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301798+0000","last_change":"2024-09-13T17:44:50.008976+0000","last_active":"2024-09-13T17:44:51.301798+0000","last_peered":"2024-09-13T17:44:51.301798+0000","last_clean":"2024-09-13T17:44:51.301798+0000","last_became_active":"2024-09-13T17:44:50.007725+0000","last_became_peered":"2024-09-13T17:44:50.007725+0000","last_unstale":"2024-09-13T17:44:51.301798+0000","last_undegraded":"2024-09-13T17:44:51.301798+0000","last_fullsized":"2024-09-13T17:44:51.301798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:43:52.006225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301596+0000","last_change":"2024-09-13T17:44:50.320797+0000","last_active":"2024-09-13T17:44:51.301596+0000","last_peered":"2024-09-13T17:44:51.301596+0000","last_clean":"2024-09-13T17:44:51.301596+0000","last_became_active":"2024-09-13T17:44:50.320440+0000","last_became_peered":"2024-09-13T17:44:50.320440+0000","last_unstale":"2024-09-13T17:44:51.301596+0000","last_undegraded":"2024-09-13T17:44:51.301596+0000","last_fullsized":"2024-09-13T17:44:51.301596+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:50.578761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312560+0000","last_change":"2024-09-13T17:44:50.015116+0000","last_active":"2024-09-13T17:44:50.312560+0000","last_peered":"2024-09-13T17:44:50.312560+0000","last_clean":"2024-09-13T17:44:50.312560+0000","last_became_active":"2024-09-13T17:44:50.014865+0000","last_became_peered":"2024-09-13T17:44:50.014865+0000","last_unstale":"2024-09-13T17:44:50.312560+0000","last_undegraded":"2024-09-13T17:44:50.312560+0000","last_fullsized":"2024-09-13T17:44:50.312560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:02.258262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.337807+0000","last_change":"2024-09-13T17:44:50.336437+0000","last_active":"2024-09-13T17:44:50.337807+0000","last_peered":"2024-09-13T17:44:50.337807+0000","last_clean":"2024-09-13T17:44:50.337807+0000","last_became_active":"2024-09-13T17:44:50.336148+0000","last_became_peered":"2024-09-13T17:44:50.336148+0000","last_unstale":"2024-09-13T17:44:50.337807+0000","last_undegraded":"2024-09-13T17:44:50.337807+0000","last_fullsized":"2024-09-13T17:44:50.337807+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:47.527759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305244+0000","last_change":"2024-09-13T17:44:50.019493+0000","last_active":"2024-09-13T17:44:50.305244+0000","last_peered":"2024-09-13T17:44:50.305244+0000","last_clean":"2024-09-13T17:44:50.305244+0000","last_became_active":"2024-09-13T17:44:50.019073+0000","last_became_peered":"2024-09-13T17:44:50.019073+0000","last_unstale":"2024-09-13T17:44:50.305244+0000","last_undegraded":"2024-09-13T17:44:50.305244+0000","last_fullsized":"2024-09-13T17:44:50.305244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:52.857441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335847+0000","last_change":"2024-09-13T17:44:50.330795+0000","last_active":"2024-09-13T17:44:50.335847+0000","last_peered":"2024-09-13T17:44:50.335847+0000","last_clean":"2024-09-13T17:44:50.335847+0000","last_became_active":"2024-09-13T17:44:50.327102+0000","last_became_peered":"2024-09-13T17:44:50.327102+0000","last_unstale":"2024-09-13T17:44:50.335847+0000","last_undegraded":"2024-09-13T17:44:50.335847+0000","last_fullsized":"2024-09-13T17:44:50.335847+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:07.560502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300603+0000","last_change":"2024-09-13T17:44:50.010422+0000","last_active":"2024-09-13T17:44:51.300603+0000","last_peered":"2024-09-13T17:44:51.300603+0000","last_clean":"2024-09-13T17:44:51.300603+0000","last_became_active":"2024-09-13T17:44:50.008092+0000","last_became_peered":"2024-09-13T17:44:50.008092+0000","last_unstale":"2024-09-13T17:44:51.300603+0000","last_undegraded":"2024-09-13T17:44:51.300603+0000","last_fullsized":"2024-09-13T17:44:51.300603+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:11.553887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348572+0000","last_change":"2024-09-13T17:44:50.344579+0000","last_active":"2024-09-13T17:44:50.348572+0000","last_peered":"2024-09-13T17:44:50.348572+0000","last_clean":"2024-09-13T17:44:50.348572+0000","last_became_active":"2024-09-13T17:44:50.343095+0000","last_became_peered":"2024-09-13T17:44:50.343095+0000","last_unstale":"2024-09-13T17:44:50.348572+0000","last_undegraded":"2024-09-13T17:44:50.348572+0000","last_fullsized":"2024-09-13T17:44:50.348572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:55.586004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.336920+0000","last_change":"2024-09-13T17:44:49.949728+0000","last_active":"2024-09-13T17:44:50.336920+0000","last_peered":"2024-09-13T17:44:50.336920+0000","last_clean":"2024-09-13T17:44:50.336920+0000","last_became_active":"2024-09-13T17:44:49.949463+0000","last_became_peered":"2024-09-13T17:44:49.949463+0000","last_unstale":"2024-09-13T17:44:50.336920+0000","last_undegraded":"2024-09-13T17:44:50.336920+0000","last_fullsized":"2024-09-13T17:44:50.336920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:33.042083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"43'141","reported_seq":335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T17:44:49.941310+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:44:49.941310+0000","last_peered":"2024-09-13T17:44:49.941310+0000","last_clean":"2024-09-13T17:44:49.941310+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:44:49.941310+0000","last_undegraded":"2024-09-13T17:44:49.941310+0000","last_fullsized":"2024-09-13T17:44:49.941310+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":217,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":217,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738406,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":569344,"data_stored":211966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40400000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738407,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1146880,"data_stored":798144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.31900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.46200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.30199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738406,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1130496,"data_stored":796102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29899999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.502},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:54.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:44:54.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.471+0000 7f781b0e5640 1 Processor -- start 2024-09-13T17:44:54.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.471+0000 7f781b0e5640 1 -- start start 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f78140a1240 0x7f7814099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78140a0e10 con 0x7f78140a1620 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78140a0210 con 0x7f78140a1240 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7813fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f78140a1620 0x7f7814135b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59246/0 (socket says 172.21.15.96:59246) 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7813fff640 1 -- 172.21.15.96:0/360538535 learned_addr learned my addr 172.21.15.96:0/360538535 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7818e81640 1 --2- 172.21.15.96:0/360538535 >> v2:172.21.15.96:3300/0 conn(0x7f78140a1240 0x7f7814099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f78137fe640 1 -- 172.21.15.96:0/360538535 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817277949 0 0) 0x7f78140a0e10 con 0x7f78140a1620 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f78137fe640 1 -- 172.21.15.96:0/360538535 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7800003530 con 0x7f78140a1620 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7818e81640 1 -- 172.21.15.96:0/360538535 >> v1:172.21.15.96:6789/0 conn(0x7f78140a1620 legacy=0x7f7814135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7818e81640 1 -- 172.21.15.96:0/360538535 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78140a2480 con 0x7f78140a1240 2024-09-13T17:44:54.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f7818e81640 1 --2- 172.21.15.96:0/360538535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814099a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f780c009d90 tx=0x7f780c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997d544c805fec99 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f78137fe640 1 -- 172.21.15.96:0/360538535 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780c0a5810 con 0x7f78140a1240 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f78137fe640 1 -- 172.21.15.96:0/360538535 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c0a5970 con 0x7f78140a1240 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f78137fe640 1 -- 172.21.15.96:0/360538535 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780c0a5c10 con 0x7f78140a1240 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- 172.21.15.96:0/360538535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 msgr2=0x7f7814099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 --2- 172.21.15.96:0/360538535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814099a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f780c009d90 tx=0x7f780c01c880 comp rx=0 tx=0).stop 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- 172.21.15.96:0/360538535 shutdown_connections 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 --2- 172.21.15.96:0/360538535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814099a60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- 172.21.15.96:0/360538535 >> 172.21.15.96:0/360538535 conn(0x7f78140a6540 msgr2=0x7f78140a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:54.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- 172.21.15.96:0/360538535 shutdown_connections 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.475+0000 7f781b0e5640 1 -- 172.21.15.96:0/360538535 wait complete. 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 Processor -- start 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 -- start start 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7814070570 con 0x7f78140a1240 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7818e81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7818e81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40560/0 (socket says 172.21.15.96:40560) 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7818e81640 1 -- 172.21.15.96:0/2029175141 learned_addr learned my addr 172.21.15.96:0/2029175141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7818e81640 1 -- 172.21.15.96:0/2029175141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780c009a70 con 0x7f78140a1240 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7818e81640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f780c0194f0 tx=0x7f780c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780c003ce0 con 0x7f78140a1240 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c003e40 con 0x7f78140a1240 2024-09-13T17:44:54.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781406f5d0 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f780c004170 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7814070780 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f780c0b4050 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7814099a60 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 0x7f77f4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f780c0e29b0 con 0x7f78140a1240 2024-09-13T17:44:54.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7813fff640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 0x7f77f4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.479+0000 7f7813fff640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 0x7f77f4037b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007fd0 tx=0x7f77fc007980 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.483+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f780c0ca760 con 0x7f78140a1240 2024-09-13T17:44:54.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.639+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 --> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f781413de40 con 0x7f77f4035690 2024-09-13T17:44:54.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.643+0000 7f78117fa640 1 -- 172.21.15.96:0/2029175141 <== mgr.4986 v2:172.21.15.96:6824/3506364525 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250052 (secure 0 0 0) 0x7f77fc0058c0 con 0x7f77f4035690 2024-09-13T17:44:54.652 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:54.653 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 msgr2=0x7f77f4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 0x7f77f4037b50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77fc007fd0 tx=0x7f77fc007980 comp rx=0 tx=0).stop 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 msgr2=0x7f7814070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f780c0194f0 tx=0x7f780c019680 comp rx=0 tx=0).stop 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 shutdown_connections 2024-09-13T17:44:54.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f77f4035690 0x7f77f4037b50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 --2- 172.21.15.96:0/2029175141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78140a1240 0x7f7814070030 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 >> 172.21.15.96:0/2029175141 conn(0x7f78140a6540 msgr2=0x7f7814132d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:54.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 shutdown_connections 2024-09-13T17:44:54.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.647+0000 7f781b0e5640 1 -- 172.21.15.96:0/2029175141 wait complete. 2024-09-13T17:44:54.669 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-13T17:44:53.596693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":261,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":96152,"kb_used_data":2780,"kb_used_omap":4,"kb_used_meta":93307,"kb_avail":283019368,"statfs":{"total":289910292480,"available":289811832832,"internally_reserved":0,"allocated":2846720,"data_stored":1806212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":95546707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.691718"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348595+0000","last_change":"2024-09-13T17:44:50.342910+0000","last_active":"2024-09-13T17:44:50.348595+0000","last_peered":"2024-09-13T17:44:50.348595+0000","last_clean":"2024-09-13T17:44:50.348595+0000","last_became_active":"2024-09-13T17:44:50.342120+0000","last_became_peered":"2024-09-13T17:44:50.342120+0000","last_unstale":"2024-09-13T17:44:50.348595+0000","last_undegraded":"2024-09-13T17:44:50.348595+0000","last_fullsized":"2024-09-13T17:44:50.348595+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:11.433421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335521+0000","last_change":"2024-09-13T17:44:50.330106+0000","last_active":"2024-09-13T17:44:50.335521+0000","last_peered":"2024-09-13T17:44:50.335521+0000","last_clean":"2024-09-13T17:44:50.335521+0000","last_became_active":"2024-09-13T17:44:50.328575+0000","last_became_peered":"2024-09-13T17:44:50.328575+0000","last_unstale":"2024-09-13T17:44:50.335521+0000","last_undegraded":"2024-09-13T17:44:50.335521+0000","last_fullsized":"2024-09-13T17:44:50.335521+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:25:30.164165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302636+0000","last_change":"2024-09-13T17:44:50.328043+0000","last_active":"2024-09-13T17:44:51.302636+0000","last_peered":"2024-09-13T17:44:51.302636+0000","last_clean":"2024-09-13T17:44:51.302636+0000","last_became_active":"2024-09-13T17:44:50.323021+0000","last_became_peered":"2024-09-13T17:44:50.323021+0000","last_unstale":"2024-09-13T17:44:51.302636+0000","last_undegraded":"2024-09-13T17:44:51.302636+0000","last_fullsized":"2024-09-13T17:44:51.302636+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:20.503150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349282+0000","last_change":"2024-09-13T17:44:50.344770+0000","last_active":"2024-09-13T17:44:50.349282+0000","last_peered":"2024-09-13T17:44:50.349282+0000","last_clean":"2024-09-13T17:44:50.349282+0000","last_became_active":"2024-09-13T17:44:50.344481+0000","last_became_peered":"2024-09-13T17:44:50.344481+0000","last_unstale":"2024-09-13T17:44:50.349282+0000","last_undegraded":"2024-09-13T17:44:50.349282+0000","last_fullsized":"2024-09-13T17:44:50.349282+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:38.778989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301481+0000","last_change":"2024-09-13T17:44:50.320652+0000","last_active":"2024-09-13T17:44:51.301481+0000","last_peered":"2024-09-13T17:44:51.301481+0000","last_clean":"2024-09-13T17:44:51.301481+0000","last_became_active":"2024-09-13T17:44:50.320149+0000","last_became_peered":"2024-09-13T17:44:50.320149+0000","last_unstale":"2024-09-13T17:44:51.301481+0000","last_undegraded":"2024-09-13T17:44:51.301481+0000","last_fullsized":"2024-09-13T17:44:51.301481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:03.443486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300909+0000","last_change":"2024-09-13T17:44:50.324956+0000","last_active":"2024-09-13T17:44:51.300909+0000","last_peered":"2024-09-13T17:44:51.300909+0000","last_clean":"2024-09-13T17:44:51.300909+0000","last_became_active":"2024-09-13T17:44:50.319923+0000","last_became_peered":"2024-09-13T17:44:50.319923+0000","last_unstale":"2024-09-13T17:44:51.300909+0000","last_undegraded":"2024-09-13T17:44:51.300909+0000","last_fullsized":"2024-09-13T17:44:51.300909+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:00:44.363201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348697+0000","last_change":"2024-09-13T17:44:50.345091+0000","last_active":"2024-09-13T17:44:50.348697+0000","last_peered":"2024-09-13T17:44:50.348697+0000","last_clean":"2024-09-13T17:44:50.348697+0000","last_became_active":"2024-09-13T17:44:50.344845+0000","last_became_peered":"2024-09-13T17:44:50.344845+0000","last_unstale":"2024-09-13T17:44:50.348697+0000","last_undegraded":"2024-09-13T17:44:50.348697+0000","last_fullsized":"2024-09-13T17:44:50.348697+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:31.116346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335983+0000","last_change":"2024-09-13T17:44:50.332460+0000","last_active":"2024-09-13T17:44:50.335983+0000","last_peered":"2024-09-13T17:44:50.335983+0000","last_clean":"2024-09-13T17:44:50.335983+0000","last_became_active":"2024-09-13T17:44:50.332067+0000","last_became_peered":"2024-09-13T17:44:50.332067+0000","last_unstale":"2024-09-13T17:44:50.335983+0000","last_undegraded":"2024-09-13T17:44:50.335983+0000","last_fullsized":"2024-09-13T17:44:50.335983+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:13:01.246561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301991+0000","last_change":"2024-09-13T17:44:50.317608+0000","last_active":"2024-09-13T17:44:51.301991+0000","last_peered":"2024-09-13T17:44:51.301991+0000","last_clean":"2024-09-13T17:44:51.301991+0000","last_became_active":"2024-09-13T17:44:50.317343+0000","last_became_peered":"2024-09-13T17:44:50.317343+0000","last_unstale":"2024-09-13T17:44:51.301991+0000","last_undegraded":"2024-09-13T17:44:51.301991+0000","last_fullsized":"2024-09-13T17:44:51.301991+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:37.835126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348620+0000","last_change":"2024-09-13T17:44:50.344641+0000","last_active":"2024-09-13T17:44:50.348620+0000","last_peered":"2024-09-13T17:44:50.348620+0000","last_clean":"2024-09-13T17:44:50.348620+0000","last_became_active":"2024-09-13T17:44:50.343162+0000","last_became_peered":"2024-09-13T17:44:50.343162+0000","last_unstale":"2024-09-13T17:44:50.348620+0000","last_undegraded":"2024-09-13T17:44:50.348620+0000","last_fullsized":"2024-09-13T17:44:50.348620+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:33.970616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346093+0000","last_change":"2024-09-13T17:44:50.341688+0000","last_active":"2024-09-13T17:44:50.346093+0000","last_peered":"2024-09-13T17:44:50.346093+0000","last_clean":"2024-09-13T17:44:50.346093+0000","last_became_active":"2024-09-13T17:44:50.341439+0000","last_became_peered":"2024-09-13T17:44:50.341439+0000","last_unstale":"2024-09-13T17:44:50.346093+0000","last_undegraded":"2024-09-13T17:44:50.346093+0000","last_fullsized":"2024-09-13T17:44:50.346093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:33:29.756925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348557+0000","last_change":"2024-09-13T17:44:50.344536+0000","last_active":"2024-09-13T17:44:50.348557+0000","last_peered":"2024-09-13T17:44:50.348557+0000","last_clean":"2024-09-13T17:44:50.348557+0000","last_became_active":"2024-09-13T17:44:50.339854+0000","last_became_peered":"2024-09-13T17:44:50.339854+0000","last_unstale":"2024-09-13T17:44:50.348557+0000","last_undegraded":"2024-09-13T17:44:50.348557+0000","last_fullsized":"2024-09-13T17:44:50.348557+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:40.121717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335950+0000","last_change":"2024-09-13T17:44:50.332266+0000","last_active":"2024-09-13T17:44:50.335950+0000","last_peered":"2024-09-13T17:44:50.335950+0000","last_clean":"2024-09-13T17:44:50.335950+0000","last_became_active":"2024-09-13T17:44:50.330114+0000","last_became_peered":"2024-09-13T17:44:50.330114+0000","last_unstale":"2024-09-13T17:44:50.335950+0000","last_undegraded":"2024-09-13T17:44:50.335950+0000","last_fullsized":"2024-09-13T17:44:50.335950+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:22:58.004541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301010+0000","last_change":"2024-09-13T17:44:50.332356+0000","last_active":"2024-09-13T17:44:51.301010+0000","last_peered":"2024-09-13T17:44:51.301010+0000","last_clean":"2024-09-13T17:44:51.301010+0000","last_became_active":"2024-09-13T17:44:50.329405+0000","last_became_peered":"2024-09-13T17:44:50.329405+0000","last_unstale":"2024-09-13T17:44:51.301010+0000","last_undegraded":"2024-09-13T17:44:51.301010+0000","last_fullsized":"2024-09-13T17:44:51.301010+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:49.465881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333087+0000","last_change":"2024-09-13T17:44:50.325133+0000","last_active":"2024-09-13T17:44:50.333087+0000","last_peered":"2024-09-13T17:44:50.333087+0000","last_clean":"2024-09-13T17:44:50.333087+0000","last_became_active":"2024-09-13T17:44:50.323005+0000","last_became_peered":"2024-09-13T17:44:50.323005+0000","last_unstale":"2024-09-13T17:44:50.333087+0000","last_undegraded":"2024-09-13T17:44:50.333087+0000","last_fullsized":"2024-09-13T17:44:50.333087+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:38.865461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300433+0000","last_change":"2024-09-13T17:44:50.317293+0000","last_active":"2024-09-13T17:44:51.300433+0000","last_peered":"2024-09-13T17:44:51.300433+0000","last_clean":"2024-09-13T17:44:51.300433+0000","last_became_active":"2024-09-13T17:44:50.316974+0000","last_became_peered":"2024-09-13T17:44:50.316974+0000","last_unstale":"2024-09-13T17:44:51.300433+0000","last_undegraded":"2024-09-13T17:44:51.300433+0000","last_fullsized":"2024-09-13T17:44:51.300433+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:50.648279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302094+0000","last_change":"2024-09-13T17:44:50.336056+0000","last_active":"2024-09-13T17:44:51.302094+0000","last_peered":"2024-09-13T17:44:51.302094+0000","last_clean":"2024-09-13T17:44:51.302094+0000","last_became_active":"2024-09-13T17:44:50.334661+0000","last_became_peered":"2024-09-13T17:44:50.334661+0000","last_unstale":"2024-09-13T17:44:51.302094+0000","last_undegraded":"2024-09-13T17:44:51.302094+0000","last_fullsized":"2024-09-13T17:44:51.302094+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:22:38.778989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348466+0000","last_change":"2024-09-13T17:44:50.342794+0000","last_active":"2024-09-13T17:44:50.348466+0000","last_peered":"2024-09-13T17:44:50.348466+0000","last_clean":"2024-09-13T17:44:50.348466+0000","last_became_active":"2024-09-13T17:44:50.342354+0000","last_became_peered":"2024-09-13T17:44:50.342354+0000","last_unstale":"2024-09-13T17:44:50.348466+0000","last_undegraded":"2024-09-13T17:44:50.348466+0000","last_fullsized":"2024-09-13T17:44:50.348466+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:19:23.266864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301990+0000","last_change":"2024-09-13T17:44:50.331952+0000","last_active":"2024-09-13T17:44:51.301990+0000","last_peered":"2024-09-13T17:44:51.301990+0000","last_clean":"2024-09-13T17:44:51.301990+0000","last_became_active":"2024-09-13T17:44:50.331369+0000","last_became_peered":"2024-09-13T17:44:50.331369+0000","last_unstale":"2024-09-13T17:44:51.301990+0000","last_undegraded":"2024-09-13T17:44:51.301990+0000","last_fullsized":"2024-09-13T17:44:51.301990+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:43:40.121717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.343000+0000","last_change":"2024-09-13T17:44:50.340821+0000","last_active":"2024-09-13T17:44:50.343000+0000","last_peered":"2024-09-13T17:44:50.343000+0000","last_clean":"2024-09-13T17:44:50.343000+0000","last_became_active":"2024-09-13T17:44:50.340546+0000","last_became_peered":"2024-09-13T17:44:50.340546+0000","last_unstale":"2024-09-13T17:44:50.343000+0000","last_undegraded":"2024-09-13T17:44:50.343000+0000","last_fullsized":"2024-09-13T17:44:50.343000+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:07.216605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.323967+0000","last_change":"2024-09-13T17:44:50.321665+0000","last_active":"2024-09-13T17:44:50.323967+0000","last_peered":"2024-09-13T17:44:50.323967+0000","last_clean":"2024-09-13T17:44:50.323967+0000","last_became_active":"2024-09-13T17:44:50.321257+0000","last_became_peered":"2024-09-13T17:44:50.321257+0000","last_unstale":"2024-09-13T17:44:50.323967+0000","last_undegraded":"2024-09-13T17:44:50.323967+0000","last_fullsized":"2024-09-13T17:44:50.323967+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:21.931915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301010+0000","last_change":"2024-09-13T17:44:50.332522+0000","last_active":"2024-09-13T17:44:51.301010+0000","last_peered":"2024-09-13T17:44:51.301010+0000","last_clean":"2024-09-13T17:44:51.301010+0000","last_became_active":"2024-09-13T17:44:50.329324+0000","last_became_peered":"2024-09-13T17:44:50.329324+0000","last_unstale":"2024-09-13T17:44:51.301010+0000","last_undegraded":"2024-09-13T17:44:51.301010+0000","last_fullsized":"2024-09-13T17:44:51.301010+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:55.586004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324542+0000","last_change":"2024-09-13T17:44:50.323392+0000","last_active":"2024-09-13T17:44:50.324542+0000","last_peered":"2024-09-13T17:44:50.324542+0000","last_clean":"2024-09-13T17:44:50.324542+0000","last_became_active":"2024-09-13T17:44:50.323154+0000","last_became_peered":"2024-09-13T17:44:50.323154+0000","last_unstale":"2024-09-13T17:44:50.324542+0000","last_undegraded":"2024-09-13T17:44:50.324542+0000","last_fullsized":"2024-09-13T17:44:50.324542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:41:55.794634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346082+0000","last_change":"2024-09-13T17:44:50.341383+0000","last_active":"2024-09-13T17:44:50.346082+0000","last_peered":"2024-09-13T17:44:50.346082+0000","last_clean":"2024-09-13T17:44:50.346082+0000","last_became_active":"2024-09-13T17:44:50.340966+0000","last_became_peered":"2024-09-13T17:44:50.340966+0000","last_unstale":"2024-09-13T17:44:50.346082+0000","last_undegraded":"2024-09-13T17:44:50.346082+0000","last_fullsized":"2024-09-13T17:44:50.346082+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:18:47.271487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326049+0000","last_change":"2024-09-13T17:44:50.320797+0000","last_active":"2024-09-13T17:44:50.326049+0000","last_peered":"2024-09-13T17:44:50.326049+0000","last_clean":"2024-09-13T17:44:50.326049+0000","last_became_active":"2024-09-13T17:44:50.320545+0000","last_became_peered":"2024-09-13T17:44:50.320545+0000","last_unstale":"2024-09-13T17:44:50.326049+0000","last_undegraded":"2024-09-13T17:44:50.326049+0000","last_fullsized":"2024-09-13T17:44:50.326049+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:10:59.117315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346177+0000","last_change":"2024-09-13T17:44:50.342612+0000","last_active":"2024-09-13T17:44:50.346177+0000","last_peered":"2024-09-13T17:44:50.346177+0000","last_clean":"2024-09-13T17:44:50.346177+0000","last_became_active":"2024-09-13T17:44:50.342168+0000","last_became_peered":"2024-09-13T17:44:50.342168+0000","last_unstale":"2024-09-13T17:44:50.346177+0000","last_undegraded":"2024-09-13T17:44:50.346177+0000","last_fullsized":"2024-09-13T17:44:50.346177+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:57:37.107184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335294+0000","last_change":"2024-09-13T17:44:50.328811+0000","last_active":"2024-09-13T17:44:50.335294+0000","last_peered":"2024-09-13T17:44:50.335294+0000","last_clean":"2024-09-13T17:44:50.335294+0000","last_became_active":"2024-09-13T17:44:50.328561+0000","last_became_peered":"2024-09-13T17:44:50.328561+0000","last_unstale":"2024-09-13T17:44:50.335294+0000","last_undegraded":"2024-09-13T17:44:50.335294+0000","last_fullsized":"2024-09-13T17:44:50.335294+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:08:36.490472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.343020+0000","last_change":"2024-09-13T17:44:50.340883+0000","last_active":"2024-09-13T17:44:50.343020+0000","last_peered":"2024-09-13T17:44:50.343020+0000","last_clean":"2024-09-13T17:44:50.343020+0000","last_became_active":"2024-09-13T17:44:50.340580+0000","last_became_peered":"2024-09-13T17:44:50.340580+0000","last_unstale":"2024-09-13T17:44:50.343020+0000","last_undegraded":"2024-09-13T17:44:50.343020+0000","last_fullsized":"2024-09-13T17:44:50.343020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:07.932090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333514+0000","last_change":"2024-09-13T17:44:50.326057+0000","last_active":"2024-09-13T17:44:50.333514+0000","last_peered":"2024-09-13T17:44:50.333514+0000","last_clean":"2024-09-13T17:44:50.333514+0000","last_became_active":"2024-09-13T17:44:50.320825+0000","last_became_peered":"2024-09-13T17:44:50.320825+0000","last_unstale":"2024-09-13T17:44:50.333514+0000","last_undegraded":"2024-09-13T17:44:50.333514+0000","last_fullsized":"2024-09-13T17:44:50.333514+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:16:49.465881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324447+0000","last_change":"2024-09-13T17:44:50.321619+0000","last_active":"2024-09-13T17:44:50.324447+0000","last_peered":"2024-09-13T17:44:50.324447+0000","last_clean":"2024-09-13T17:44:50.324447+0000","last_became_active":"2024-09-13T17:44:50.321300+0000","last_became_peered":"2024-09-13T17:44:50.321300+0000","last_unstale":"2024-09-13T17:44:50.324447+0000","last_undegraded":"2024-09-13T17:44:50.324447+0000","last_fullsized":"2024-09-13T17:44:50.324447+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:50.641774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.325430+0000","last_change":"2024-09-13T17:44:50.322775+0000","last_active":"2024-09-13T17:44:50.325430+0000","last_peered":"2024-09-13T17:44:50.325430+0000","last_clean":"2024-09-13T17:44:50.325430+0000","last_became_active":"2024-09-13T17:44:50.322517+0000","last_became_peered":"2024-09-13T17:44:50.322517+0000","last_unstale":"2024-09-13T17:44:50.325430+0000","last_undegraded":"2024-09-13T17:44:50.325430+0000","last_fullsized":"2024-09-13T17:44:50.325430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:22:44.502496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324572+0000","last_change":"2024-09-13T17:44:50.322670+0000","last_active":"2024-09-13T17:44:50.324572+0000","last_peered":"2024-09-13T17:44:50.324572+0000","last_clean":"2024-09-13T17:44:50.324572+0000","last_became_active":"2024-09-13T17:44:50.322159+0000","last_became_peered":"2024-09-13T17:44:50.322159+0000","last_unstale":"2024-09-13T17:44:50.324572+0000","last_undegraded":"2024-09-13T17:44:50.324572+0000","last_fullsized":"2024-09-13T17:44:50.324572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:23:56.912889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302861+0000","last_change":"2024-09-13T17:44:50.319795+0000","last_active":"2024-09-13T17:44:51.302861+0000","last_peered":"2024-09-13T17:44:51.302861+0000","last_clean":"2024-09-13T17:44:51.302861+0000","last_became_active":"2024-09-13T17:44:50.319553+0000","last_became_peered":"2024-09-13T17:44:50.319553+0000","last_unstale":"2024-09-13T17:44:51.302861+0000","last_undegraded":"2024-09-13T17:44:51.302861+0000","last_fullsized":"2024-09-13T17:44:51.302861+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:56:44.728477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1f","version":"55'4","reported_seq":15,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302480+0000","last_change":"2024-09-13T17:44:50.010193+0000","last_active":"2024-09-13T17:44:51.302480+0000","last_peered":"2024-09-13T17:44:51.302480+0000","last_clean":"2024-09-13T17:44:51.302480+0000","last_became_active":"2024-09-13T17:44:50.009934+0000","last_became_peered":"2024-09-13T17:44:50.009934+0000","last_unstale":"2024-09-13T17:44:51.302480+0000","last_undegraded":"2024-09-13T17:44:51.302480+0000","last_fullsized":"2024-09-13T17:44:51.302480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:07:21.419980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302090+0000","last_change":"2024-09-13T17:44:50.336829+0000","last_active":"2024-09-13T17:44:51.302090+0000","last_peered":"2024-09-13T17:44:51.302090+0000","last_clean":"2024-09-13T17:44:51.302090+0000","last_became_active":"2024-09-13T17:44:50.336470+0000","last_became_peered":"2024-09-13T17:44:50.336470+0000","last_unstale":"2024-09-13T17:44:51.302090+0000","last_undegraded":"2024-09-13T17:44:51.302090+0000","last_fullsized":"2024-09-13T17:44:51.302090+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:30.343357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.340244+0000","last_change":"2024-09-13T17:44:50.019324+0000","last_active":"2024-09-13T17:44:50.340244+0000","last_peered":"2024-09-13T17:44:50.340244+0000","last_clean":"2024-09-13T17:44:50.340244+0000","last_became_active":"2024-09-13T17:44:50.019017+0000","last_became_peered":"2024-09-13T17:44:50.019017+0000","last_unstale":"2024-09-13T17:44:50.340244+0000","last_undegraded":"2024-09-13T17:44:50.340244+0000","last_fullsized":"2024-09-13T17:44:50.340244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:41.923832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301742+0000","last_change":"2024-09-13T17:44:50.322892+0000","last_active":"2024-09-13T17:44:51.301742+0000","last_peered":"2024-09-13T17:44:51.301742+0000","last_clean":"2024-09-13T17:44:51.301742+0000","last_became_active":"2024-09-13T17:44:50.318045+0000","last_became_peered":"2024-09-13T17:44:50.318045+0000","last_unstale":"2024-09-13T17:44:51.301742+0000","last_undegraded":"2024-09-13T17:44:51.301742+0000","last_fullsized":"2024-09-13T17:44:51.301742+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:01:09.358687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"55'2","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302607+0000","last_change":"2024-09-13T17:44:50.010620+0000","last_active":"2024-09-13T17:44:51.302607+0000","last_peered":"2024-09-13T17:44:51.302607+0000","last_clean":"2024-09-13T17:44:51.302607+0000","last_became_active":"2024-09-13T17:44:50.009775+0000","last_became_peered":"2024-09-13T17:44:50.009775+0000","last_unstale":"2024-09-13T17:44:51.302607+0000","last_undegraded":"2024-09-13T17:44:51.302607+0000","last_fullsized":"2024-09-13T17:44:51.302607+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:09:45.360880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.333653+0000","last_change":"2024-09-13T17:44:50.326684+0000","last_active":"2024-09-13T17:44:50.333653+0000","last_peered":"2024-09-13T17:44:50.333653+0000","last_clean":"2024-09-13T17:44:50.333653+0000","last_became_active":"2024-09-13T17:44:50.326364+0000","last_became_peered":"2024-09-13T17:44:50.326364+0000","last_unstale":"2024-09-13T17:44:50.333653+0000","last_undegraded":"2024-09-13T17:44:50.333653+0000","last_fullsized":"2024-09-13T17:44:50.333653+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:39.872446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"55'2","reported_seq":13,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301913+0000","last_change":"2024-09-13T17:44:50.008813+0000","last_active":"2024-09-13T17:44:51.301913+0000","last_peered":"2024-09-13T17:44:51.301913+0000","last_clean":"2024-09-13T17:44:51.301913+0000","last_became_active":"2024-09-13T17:44:50.008496+0000","last_became_peered":"2024-09-13T17:44:50.008496+0000","last_unstale":"2024-09-13T17:44:51.301913+0000","last_undegraded":"2024-09-13T17:44:51.301913+0000","last_fullsized":"2024-09-13T17:44:51.301913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:50.066142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301122+0000","last_change":"2024-09-13T17:44:50.328909+0000","last_active":"2024-09-13T17:44:51.301122+0000","last_peered":"2024-09-13T17:44:51.301122+0000","last_clean":"2024-09-13T17:44:51.301122+0000","last_became_active":"2024-09-13T17:44:50.320247+0000","last_became_peered":"2024-09-13T17:44:50.320247+0000","last_unstale":"2024-09-13T17:44:51.301122+0000","last_undegraded":"2024-09-13T17:44:51.301122+0000","last_fullsized":"2024-09-13T17:44:51.301122+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:31:03.823794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1b","version":"55'1","reported_seq":12,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301691+0000","last_change":"2024-09-13T17:44:50.009366+0000","last_active":"2024-09-13T17:44:51.301691+0000","last_peered":"2024-09-13T17:44:51.301691+0000","last_clean":"2024-09-13T17:44:51.301691+0000","last_became_active":"2024-09-13T17:44:50.009055+0000","last_became_peered":"2024-09-13T17:44:50.009055+0000","last_unstale":"2024-09-13T17:44:51.301691+0000","last_undegraded":"2024-09-13T17:44:51.301691+0000","last_fullsized":"2024-09-13T17:44:51.301691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:15.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301430+0000","last_change":"2024-09-13T17:44:50.336546+0000","last_active":"2024-09-13T17:44:51.301430+0000","last_peered":"2024-09-13T17:44:51.301430+0000","last_clean":"2024-09-13T17:44:51.301430+0000","last_became_active":"2024-09-13T17:44:50.336264+0000","last_became_peered":"2024-09-13T17:44:50.336264+0000","last_unstale":"2024-09-13T17:44:51.301430+0000","last_undegraded":"2024-09-13T17:44:51.301430+0000","last_fullsized":"2024-09-13T17:44:51.301430+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:53:31.116346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.347701+0000","last_change":"2024-09-13T17:44:50.016616+0000","last_active":"2024-09-13T17:44:50.347701+0000","last_peered":"2024-09-13T17:44:50.347701+0000","last_clean":"2024-09-13T17:44:50.347701+0000","last_became_active":"2024-09-13T17:44:50.016359+0000","last_became_peered":"2024-09-13T17:44:50.016359+0000","last_unstale":"2024-09-13T17:44:50.347701+0000","last_undegraded":"2024-09-13T17:44:50.347701+0000","last_fullsized":"2024-09-13T17:44:50.347701+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:50.066142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300491+0000","last_change":"2024-09-13T17:44:50.332439+0000","last_active":"2024-09-13T17:44:51.300491+0000","last_peered":"2024-09-13T17:44:51.300491+0000","last_clean":"2024-09-13T17:44:51.300491+0000","last_became_active":"2024-09-13T17:44:50.332037+0000","last_became_peered":"2024-09-13T17:44:50.332037+0000","last_unstale":"2024-09-13T17:44:51.300491+0000","last_undegraded":"2024-09-13T17:44:51.300491+0000","last_fullsized":"2024-09-13T17:44:51.300491+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:27:11.433421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.311633+0000","last_change":"2024-09-13T17:44:50.013828+0000","last_active":"2024-09-13T17:44:50.311633+0000","last_peered":"2024-09-13T17:44:50.311633+0000","last_clean":"2024-09-13T17:44:50.311633+0000","last_became_active":"2024-09-13T17:44:50.013573+0000","last_became_peered":"2024-09-13T17:44:50.013573+0000","last_unstale":"2024-09-13T17:44:50.311633+0000","last_undegraded":"2024-09-13T17:44:50.311633+0000","last_fullsized":"2024-09-13T17:44:50.311633+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:01:42.012204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335868+0000","last_change":"2024-09-13T17:44:50.331174+0000","last_active":"2024-09-13T17:44:50.335868+0000","last_peered":"2024-09-13T17:44:50.335868+0000","last_clean":"2024-09-13T17:44:50.335868+0000","last_became_active":"2024-09-13T17:44:50.330459+0000","last_became_peered":"2024-09-13T17:44:50.330459+0000","last_unstale":"2024-09-13T17:44:50.335868+0000","last_undegraded":"2024-09-13T17:44:50.335868+0000","last_fullsized":"2024-09-13T17:44:50.335868+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:55:17.418015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300568+0000","last_change":"2024-09-13T17:44:50.010224+0000","last_active":"2024-09-13T17:44:51.300568+0000","last_peered":"2024-09-13T17:44:51.300568+0000","last_clean":"2024-09-13T17:44:51.300568+0000","last_became_active":"2024-09-13T17:44:50.008628+0000","last_became_peered":"2024-09-13T17:44:50.008628+0000","last_unstale":"2024-09-13T17:44:51.300568+0000","last_undegraded":"2024-09-13T17:44:51.300568+0000","last_fullsized":"2024-09-13T17:44:51.300568+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:25:21.595209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324628+0000","last_change":"2024-09-13T17:44:50.322139+0000","last_active":"2024-09-13T17:44:50.324628+0000","last_peered":"2024-09-13T17:44:50.324628+0000","last_clean":"2024-09-13T17:44:50.324628+0000","last_became_active":"2024-09-13T17:44:50.321994+0000","last_became_peered":"2024-09-13T17:44:50.321994+0000","last_unstale":"2024-09-13T17:44:50.324628+0000","last_undegraded":"2024-09-13T17:44:50.324628+0000","last_fullsized":"2024-09-13T17:44:50.324628+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:24:40.580340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305889+0000","last_change":"2024-09-13T17:44:49.947282+0000","last_active":"2024-09-13T17:44:50.305889+0000","last_peered":"2024-09-13T17:44:50.305889+0000","last_clean":"2024-09-13T17:44:50.305889+0000","last_became_active":"2024-09-13T17:44:49.947009+0000","last_became_peered":"2024-09-13T17:44:49.947009+0000","last_unstale":"2024-09-13T17:44:50.305889+0000","last_undegraded":"2024-09-13T17:44:50.305889+0000","last_fullsized":"2024-09-13T17:44:50.305889+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:31.656356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335955+0000","last_change":"2024-09-13T17:44:50.330889+0000","last_active":"2024-09-13T17:44:50.335955+0000","last_peered":"2024-09-13T17:44:50.335955+0000","last_clean":"2024-09-13T17:44:50.335955+0000","last_became_active":"2024-09-13T17:44:50.330660+0000","last_became_peered":"2024-09-13T17:44:50.330660+0000","last_unstale":"2024-09-13T17:44:50.335955+0000","last_undegraded":"2024-09-13T17:44:50.335955+0000","last_fullsized":"2024-09-13T17:44:50.335955+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:04:47.640483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.306472+0000","last_change":"2024-09-13T17:44:50.019871+0000","last_active":"2024-09-13T17:44:50.306472+0000","last_peered":"2024-09-13T17:44:50.306472+0000","last_clean":"2024-09-13T17:44:50.306472+0000","last_became_active":"2024-09-13T17:44:50.019125+0000","last_became_peered":"2024-09-13T17:44:50.019125+0000","last_unstale":"2024-09-13T17:44:50.306472+0000","last_undegraded":"2024-09-13T17:44:50.306472+0000","last_fullsized":"2024-09-13T17:44:50.306472+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:31:58.749723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335492+0000","last_change":"2024-09-13T17:44:50.329956+0000","last_active":"2024-09-13T17:44:50.335492+0000","last_peered":"2024-09-13T17:44:50.335492+0000","last_clean":"2024-09-13T17:44:50.335492+0000","last_became_active":"2024-09-13T17:44:50.328065+0000","last_became_peered":"2024-09-13T17:44:50.328065+0000","last_unstale":"2024-09-13T17:44:50.335492+0000","last_undegraded":"2024-09-13T17:44:50.335492+0000","last_fullsized":"2024-09-13T17:44:50.335492+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:39:34.070976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.15","version":"55'3","reported_seq":13,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.341223+0000","last_change":"2024-09-13T17:44:50.018014+0000","last_active":"2024-09-13T17:44:50.341223+0000","last_peered":"2024-09-13T17:44:50.341223+0000","last_clean":"2024-09-13T17:44:50.341223+0000","last_became_active":"2024-09-13T17:44:50.017758+0000","last_became_peered":"2024-09-13T17:44:50.017758+0000","last_unstale":"2024-09-13T17:44:50.341223+0000","last_undegraded":"2024-09-13T17:44:50.341223+0000","last_fullsized":"2024-09-13T17:44:50.341223+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:59.256249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302117+0000","last_change":"2024-09-13T17:44:50.331791+0000","last_active":"2024-09-13T17:44:51.302117+0000","last_peered":"2024-09-13T17:44:51.302117+0000","last_clean":"2024-09-13T17:44:51.302117+0000","last_became_active":"2024-09-13T17:44:50.328573+0000","last_became_peered":"2024-09-13T17:44:50.328573+0000","last_unstale":"2024-09-13T17:44:51.302117+0000","last_undegraded":"2024-09-13T17:44:51.302117+0000","last_fullsized":"2024-09-13T17:44:51.302117+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:16.762863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"55'6","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349747+0000","last_change":"2024-09-13T17:44:50.014447+0000","last_active":"2024-09-13T17:44:50.349747+0000","last_peered":"2024-09-13T17:44:50.349747+0000","last_clean":"2024-09-13T17:44:50.349747+0000","last_became_active":"2024-09-13T17:44:50.014096+0000","last_became_peered":"2024-09-13T17:44:50.014096+0000","last_unstale":"2024-09-13T17:44:50.349747+0000","last_undegraded":"2024-09-13T17:44:50.349747+0000","last_fullsized":"2024-09-13T17:44:50.349747+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:50:55.839610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301214+0000","last_change":"2024-09-13T17:44:50.329075+0000","last_active":"2024-09-13T17:44:51.301214+0000","last_peered":"2024-09-13T17:44:51.301214+0000","last_clean":"2024-09-13T17:44:51.301214+0000","last_became_active":"2024-09-13T17:44:50.320531+0000","last_became_peered":"2024-09-13T17:44:50.320531+0000","last_unstale":"2024-09-13T17:44:51.301214+0000","last_undegraded":"2024-09-13T17:44:51.301214+0000","last_fullsized":"2024-09-13T17:44:51.301214+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:32:37.332944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.347530+0000","last_change":"2024-09-13T17:44:50.013584+0000","last_active":"2024-09-13T17:44:50.347530+0000","last_peered":"2024-09-13T17:44:50.347530+0000","last_clean":"2024-09-13T17:44:50.347530+0000","last_became_active":"2024-09-13T17:44:50.013319+0000","last_became_peered":"2024-09-13T17:44:50.013319+0000","last_unstale":"2024-09-13T17:44:50.347530+0000","last_undegraded":"2024-09-13T17:44:50.347530+0000","last_fullsized":"2024-09-13T17:44:50.347530+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:43:52.006225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326198+0000","last_change":"2024-09-13T17:44:50.324064+0000","last_active":"2024-09-13T17:44:50.326198+0000","last_peered":"2024-09-13T17:44:50.326198+0000","last_clean":"2024-09-13T17:44:50.326198+0000","last_became_active":"2024-09-13T17:44:50.323879+0000","last_became_peered":"2024-09-13T17:44:50.323879+0000","last_unstale":"2024-09-13T17:44:50.326198+0000","last_undegraded":"2024-09-13T17:44:50.326198+0000","last_fullsized":"2024-09-13T17:44:50.326198+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:16.762863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312470+0000","last_change":"2024-09-13T17:44:50.016629+0000","last_active":"2024-09-13T17:44:50.312470+0000","last_peered":"2024-09-13T17:44:50.312470+0000","last_clean":"2024-09-13T17:44:50.312470+0000","last_became_active":"2024-09-13T17:44:50.016072+0000","last_became_peered":"2024-09-13T17:44:50.016072+0000","last_unstale":"2024-09-13T17:44:50.312470+0000","last_undegraded":"2024-09-13T17:44:50.312470+0000","last_fullsized":"2024-09-13T17:44:50.312470+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:07:21.419980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346069+0000","last_change":"2024-09-13T17:44:50.341042+0000","last_active":"2024-09-13T17:44:50.346069+0000","last_peered":"2024-09-13T17:44:50.346069+0000","last_clean":"2024-09-13T17:44:50.346069+0000","last_became_active":"2024-09-13T17:44:50.340617+0000","last_became_peered":"2024-09-13T17:44:50.340617+0000","last_unstale":"2024-09-13T17:44:50.346069+0000","last_undegraded":"2024-09-13T17:44:50.346069+0000","last_fullsized":"2024-09-13T17:44:50.346069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:08.078983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.311724+0000","last_change":"2024-09-13T17:44:49.946367+0000","last_active":"2024-09-13T17:44:50.311724+0000","last_peered":"2024-09-13T17:44:50.311724+0000","last_clean":"2024-09-13T17:44:50.311724+0000","last_became_active":"2024-09-13T17:44:49.943792+0000","last_became_peered":"2024-09-13T17:44:49.943792+0000","last_unstale":"2024-09-13T17:44:50.311724+0000","last_undegraded":"2024-09-13T17:44:50.311724+0000","last_fullsized":"2024-09-13T17:44:50.311724+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:52.214852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301144+0000","last_change":"2024-09-13T17:44:50.323169+0000","last_active":"2024-09-13T17:44:51.301144+0000","last_peered":"2024-09-13T17:44:51.301144+0000","last_clean":"2024-09-13T17:44:51.301144+0000","last_became_active":"2024-09-13T17:44:50.322913+0000","last_became_peered":"2024-09-13T17:44:50.322913+0000","last_unstale":"2024-09-13T17:44:51.301144+0000","last_undegraded":"2024-09-13T17:44:51.301144+0000","last_fullsized":"2024-09-13T17:44:51.301144+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:51:02.511044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"55'8","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348180+0000","last_change":"2024-09-13T17:44:50.018798+0000","last_active":"2024-09-13T17:44:50.348180+0000","last_peered":"2024-09-13T17:44:50.348180+0000","last_clean":"2024-09-13T17:44:50.348180+0000","last_became_active":"2024-09-13T17:44:50.018492+0000","last_became_peered":"2024-09-13T17:44:50.018492+0000","last_unstale":"2024-09-13T17:44:50.348180+0000","last_undegraded":"2024-09-13T17:44:50.348180+0000","last_fullsized":"2024-09-13T17:44:50.348180+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:51:15.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335988+0000","last_change":"2024-09-13T17:44:50.333401+0000","last_active":"2024-09-13T17:44:50.335988+0000","last_peered":"2024-09-13T17:44:50.335988+0000","last_clean":"2024-09-13T17:44:50.335988+0000","last_became_active":"2024-09-13T17:44:50.333129+0000","last_became_peered":"2024-09-13T17:44:50.333129+0000","last_unstale":"2024-09-13T17:44:50.335988+0000","last_undegraded":"2024-09-13T17:44:50.335988+0000","last_fullsized":"2024-09-13T17:44:50.335988+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:38:04.136915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.345041+0000","last_change":"2024-09-13T17:44:50.016270+0000","last_active":"2024-09-13T17:44:50.345041+0000","last_peered":"2024-09-13T17:44:50.345041+0000","last_clean":"2024-09-13T17:44:50.345041+0000","last_became_active":"2024-09-13T17:44:50.016025+0000","last_became_peered":"2024-09-13T17:44:50.016025+0000","last_unstale":"2024-09-13T17:44:50.345041+0000","last_undegraded":"2024-09-13T17:44:50.345041+0000","last_fullsized":"2024-09-13T17:44:50.345041+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:10:52.332294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335939+0000","last_change":"2024-09-13T17:44:50.330830+0000","last_active":"2024-09-13T17:44:50.335939+0000","last_peered":"2024-09-13T17:44:50.335939+0000","last_clean":"2024-09-13T17:44:50.335939+0000","last_became_active":"2024-09-13T17:44:50.330588+0000","last_became_peered":"2024-09-13T17:44:50.330588+0000","last_unstale":"2024-09-13T17:44:50.335939+0000","last_undegraded":"2024-09-13T17:44:50.335939+0000","last_fullsized":"2024-09-13T17:44:50.335939+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:37.132544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.328872+0000","last_change":"2024-09-13T17:44:49.947334+0000","last_active":"2024-09-13T17:44:50.328872+0000","last_peered":"2024-09-13T17:44:50.328872+0000","last_clean":"2024-09-13T17:44:50.328872+0000","last_became_active":"2024-09-13T17:44:49.946893+0000","last_became_peered":"2024-09-13T17:44:49.946893+0000","last_unstale":"2024-09-13T17:44:50.328872+0000","last_undegraded":"2024-09-13T17:44:50.328872+0000","last_fullsized":"2024-09-13T17:44:50.328872+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:09:49.638098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.349264+0000","last_change":"2024-09-13T17:44:50.344731+0000","last_active":"2024-09-13T17:44:50.349264+0000","last_peered":"2024-09-13T17:44:50.349264+0000","last_clean":"2024-09-13T17:44:50.349264+0000","last_became_active":"2024-09-13T17:44:50.340212+0000","last_became_peered":"2024-09-13T17:44:50.340212+0000","last_unstale":"2024-09-13T17:44:50.349264+0000","last_undegraded":"2024-09-13T17:44:50.349264+0000","last_fullsized":"2024-09-13T17:44:50.349264+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:02.534122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"55'1","reported_seq":11,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.341240+0000","last_change":"2024-09-13T17:44:50.017970+0000","last_active":"2024-09-13T17:44:50.341240+0000","last_peered":"2024-09-13T17:44:50.341240+0000","last_clean":"2024-09-13T17:44:50.341240+0000","last_became_active":"2024-09-13T17:44:50.017421+0000","last_became_peered":"2024-09-13T17:44:50.017421+0000","last_unstale":"2024-09-13T17:44:50.341240+0000","last_undegraded":"2024-09-13T17:44:50.341240+0000","last_fullsized":"2024-09-13T17:44:50.341240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:58:07.964647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302545+0000","last_change":"2024-09-13T17:44:50.331615+0000","last_active":"2024-09-13T17:44:51.302545+0000","last_peered":"2024-09-13T17:44:51.302545+0000","last_clean":"2024-09-13T17:44:51.302545+0000","last_became_active":"2024-09-13T17:44:50.328279+0000","last_became_peered":"2024-09-13T17:44:50.328279+0000","last_unstale":"2024-09-13T17:44:51.302545+0000","last_undegraded":"2024-09-13T17:44:51.302545+0000","last_fullsized":"2024-09-13T17:44:51.302545+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:39.872446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301825+0000","last_change":"2024-09-13T17:44:50.008833+0000","last_active":"2024-09-13T17:44:51.301825+0000","last_peered":"2024-09-13T17:44:51.301825+0000","last_clean":"2024-09-13T17:44:51.301825+0000","last_became_active":"2024-09-13T17:44:50.008509+0000","last_became_peered":"2024-09-13T17:44:50.008509+0000","last_unstale":"2024-09-13T17:44:51.301825+0000","last_undegraded":"2024-09-13T17:44:51.301825+0000","last_fullsized":"2024-09-13T17:44:51.301825+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:33.042083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301688+0000","last_change":"2024-09-13T17:44:50.336084+0000","last_active":"2024-09-13T17:44:51.301688+0000","last_peered":"2024-09-13T17:44:51.301688+0000","last_clean":"2024-09-13T17:44:51.301688+0000","last_became_active":"2024-09-13T17:44:50.334621+0000","last_became_peered":"2024-09-13T17:44:50.334621+0000","last_unstale":"2024-09-13T17:44:51.301688+0000","last_undegraded":"2024-09-13T17:44:51.301688+0000","last_fullsized":"2024-09-13T17:44:51.301688+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:48:02.534122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301280+0000","last_change":"2024-09-13T17:44:50.012564+0000","last_active":"2024-09-13T17:44:51.301280+0000","last_peered":"2024-09-13T17:44:51.301280+0000","last_clean":"2024-09-13T17:44:51.301280+0000","last_became_active":"2024-09-13T17:44:50.011951+0000","last_became_peered":"2024-09-13T17:44:50.011951+0000","last_unstale":"2024-09-13T17:44:51.301280+0000","last_undegraded":"2024-09-13T17:44:51.301280+0000","last_fullsized":"2024-09-13T17:44:51.301280+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:13:52.214852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300528+0000","last_change":"2024-09-13T17:44:50.332068+0000","last_active":"2024-09-13T17:44:51.300528+0000","last_peered":"2024-09-13T17:44:51.300528+0000","last_clean":"2024-09-13T17:44:51.300528+0000","last_became_active":"2024-09-13T17:44:50.331457+0000","last_became_peered":"2024-09-13T17:44:50.331457+0000","last_unstale":"2024-09-13T17:44:51.300528+0000","last_undegraded":"2024-09-13T17:44:51.300528+0000","last_fullsized":"2024-09-13T17:44:51.300528+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:47:33.970616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305135+0000","last_change":"2024-09-13T17:44:50.019696+0000","last_active":"2024-09-13T17:44:50.305135+0000","last_peered":"2024-09-13T17:44:50.305135+0000","last_clean":"2024-09-13T17:44:50.305135+0000","last_became_active":"2024-09-13T17:44:50.019417+0000","last_became_peered":"2024-09-13T17:44:50.019417+0000","last_unstale":"2024-09-13T17:44:50.305135+0000","last_undegraded":"2024-09-13T17:44:50.305135+0000","last_fullsized":"2024-09-13T17:44:50.305135+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:42:51.953778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301305+0000","last_change":"2024-09-13T17:44:50.327682+0000","last_active":"2024-09-13T17:44:51.301305+0000","last_peered":"2024-09-13T17:44:51.301305+0000","last_clean":"2024-09-13T17:44:51.301305+0000","last_became_active":"2024-09-13T17:44:50.327422+0000","last_became_peered":"2024-09-13T17:44:50.327422+0000","last_unstale":"2024-09-13T17:44:51.301305+0000","last_undegraded":"2024-09-13T17:44:51.301305+0000","last_fullsized":"2024-09-13T17:44:51.301305+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:10:59.117315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.304168+0000","last_change":"2024-09-13T17:44:50.017876+0000","last_active":"2024-09-13T17:44:50.304168+0000","last_peered":"2024-09-13T17:44:50.304168+0000","last_clean":"2024-09-13T17:44:50.304168+0000","last_became_active":"2024-09-13T17:44:50.017576+0000","last_became_peered":"2024-09-13T17:44:50.017576+0000","last_unstale":"2024-09-13T17:44:50.304168+0000","last_undegraded":"2024-09-13T17:44:50.304168+0000","last_fullsized":"2024-09-13T17:44:50.304168+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:11:32.395758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324992+0000","last_change":"2024-09-13T17:44:50.322119+0000","last_active":"2024-09-13T17:44:50.324992+0000","last_peered":"2024-09-13T17:44:50.324992+0000","last_clean":"2024-09-13T17:44:50.324992+0000","last_became_active":"2024-09-13T17:44:50.321856+0000","last_became_peered":"2024-09-13T17:44:50.321856+0000","last_unstale":"2024-09-13T17:44:50.324992+0000","last_undegraded":"2024-09-13T17:44:50.324992+0000","last_fullsized":"2024-09-13T17:44:50.324992+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:41:57.254892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335849+0000","last_change":"2024-09-13T17:44:49.945926+0000","last_active":"2024-09-13T17:44:50.335849+0000","last_peered":"2024-09-13T17:44:50.335849+0000","last_clean":"2024-09-13T17:44:50.335849+0000","last_became_active":"2024-09-13T17:44:49.945490+0000","last_became_peered":"2024-09-13T17:44:49.945490+0000","last_unstale":"2024-09-13T17:44:50.335849+0000","last_undegraded":"2024-09-13T17:44:50.335849+0000","last_fullsized":"2024-09-13T17:44:50.335849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:26:50.208962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.326028+0000","last_change":"2024-09-13T17:44:50.320331+0000","last_active":"2024-09-13T17:44:50.326028+0000","last_peered":"2024-09-13T17:44:50.326028+0000","last_clean":"2024-09-13T17:44:50.326028+0000","last_became_active":"2024-09-13T17:44:50.320061+0000","last_became_peered":"2024-09-13T17:44:50.320061+0000","last_unstale":"2024-09-13T17:44:50.326028+0000","last_undegraded":"2024-09-13T17:44:50.326028+0000","last_fullsized":"2024-09-13T17:44:50.326028+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:00:51.647187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312379+0000","last_change":"2024-09-13T17:44:49.944249+0000","last_active":"2024-09-13T17:44:50.312379+0000","last_peered":"2024-09-13T17:44:50.312379+0000","last_clean":"2024-09-13T17:44:50.312379+0000","last_became_active":"2024-09-13T17:44:49.943968+0000","last_became_peered":"2024-09-13T17:44:49.943968+0000","last_unstale":"2024-09-13T17:44:50.312379+0000","last_undegraded":"2024-09-13T17:44:50.312379+0000","last_fullsized":"2024-09-13T17:44:50.312379+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:37.117449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.324481+0000","last_change":"2024-09-13T17:44:50.322659+0000","last_active":"2024-09-13T17:44:50.324481+0000","last_peered":"2024-09-13T17:44:50.324481+0000","last_clean":"2024-09-13T17:44:50.324481+0000","last_became_active":"2024-09-13T17:44:50.322107+0000","last_became_peered":"2024-09-13T17:44:50.322107+0000","last_unstale":"2024-09-13T17:44:50.324481+0000","last_undegraded":"2024-09-13T17:44:50.324481+0000","last_fullsized":"2024-09-13T17:44:50.324481+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:16:35.827537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.302656+0000","last_change":"2024-09-13T17:44:50.012895+0000","last_active":"2024-09-13T17:44:51.302656+0000","last_peered":"2024-09-13T17:44:51.302656+0000","last_clean":"2024-09-13T17:44:51.302656+0000","last_became_active":"2024-09-13T17:44:50.012644+0000","last_became_peered":"2024-09-13T17:44:50.012644+0000","last_unstale":"2024-09-13T17:44:51.302656+0000","last_undegraded":"2024-09-13T17:44:51.302656+0000","last_fullsized":"2024-09-13T17:44:51.302656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:18:37.117449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.346350+0000","last_change":"2024-09-13T17:44:50.342649+0000","last_active":"2024-09-13T17:44:50.346350+0000","last_peered":"2024-09-13T17:44:50.346350+0000","last_clean":"2024-09-13T17:44:50.346350+0000","last_became_active":"2024-09-13T17:44:50.339607+0000","last_became_peered":"2024-09-13T17:44:50.339607+0000","last_unstale":"2024-09-13T17:44:50.346350+0000","last_undegraded":"2024-09-13T17:44:50.346350+0000","last_fullsized":"2024-09-13T17:44:50.346350+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:08:30.343357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305771+0000","last_change":"2024-09-13T17:44:49.945906+0000","last_active":"2024-09-13T17:44:50.305771+0000","last_peered":"2024-09-13T17:44:50.305771+0000","last_clean":"2024-09-13T17:44:50.305771+0000","last_became_active":"2024-09-13T17:44:49.945059+0000","last_became_peered":"2024-09-13T17:44:49.945059+0000","last_unstale":"2024-09-13T17:44:50.305771+0000","last_undegraded":"2024-09-13T17:44:50.305771+0000","last_fullsized":"2024-09-13T17:44:50.305771+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:40:34.030988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301461+0000","last_change":"2024-09-13T17:44:50.331088+0000","last_active":"2024-09-13T17:44:51.301461+0000","last_peered":"2024-09-13T17:44:51.301461+0000","last_clean":"2024-09-13T17:44:51.301461+0000","last_became_active":"2024-09-13T17:44:50.329057+0000","last_became_peered":"2024-09-13T17:44:50.329057+0000","last_unstale":"2024-09-13T17:44:51.301461+0000","last_undegraded":"2024-09-13T17:44:51.301461+0000","last_fullsized":"2024-09-13T17:44:51.301461+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:05:49.880699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301798+0000","last_change":"2024-09-13T17:44:50.008976+0000","last_active":"2024-09-13T17:44:51.301798+0000","last_peered":"2024-09-13T17:44:51.301798+0000","last_clean":"2024-09-13T17:44:51.301798+0000","last_became_active":"2024-09-13T17:44:50.007725+0000","last_became_peered":"2024-09-13T17:44:50.007725+0000","last_unstale":"2024-09-13T17:44:51.301798+0000","last_undegraded":"2024-09-13T17:44:51.301798+0000","last_fullsized":"2024-09-13T17:44:51.301798+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:43:52.006225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.301596+0000","last_change":"2024-09-13T17:44:50.320797+0000","last_active":"2024-09-13T17:44:51.301596+0000","last_peered":"2024-09-13T17:44:51.301596+0000","last_clean":"2024-09-13T17:44:51.301596+0000","last_became_active":"2024-09-13T17:44:50.320440+0000","last_became_peered":"2024-09-13T17:44:50.320440+0000","last_unstale":"2024-09-13T17:44:51.301596+0000","last_undegraded":"2024-09-13T17:44:51.301596+0000","last_fullsized":"2024-09-13T17:44:51.301596+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:51:50.578761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.312560+0000","last_change":"2024-09-13T17:44:50.015116+0000","last_active":"2024-09-13T17:44:50.312560+0000","last_peered":"2024-09-13T17:44:50.312560+0000","last_clean":"2024-09-13T17:44:50.312560+0000","last_became_active":"2024-09-13T17:44:50.014865+0000","last_became_peered":"2024-09-13T17:44:50.014865+0000","last_unstale":"2024-09-13T17:44:50.312560+0000","last_undegraded":"2024-09-13T17:44:50.312560+0000","last_fullsized":"2024-09-13T17:44:50.312560+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:49:02.258262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.337807+0000","last_change":"2024-09-13T17:44:50.336437+0000","last_active":"2024-09-13T17:44:50.337807+0000","last_peered":"2024-09-13T17:44:50.337807+0000","last_clean":"2024-09-13T17:44:50.337807+0000","last_became_active":"2024-09-13T17:44:50.336148+0000","last_became_peered":"2024-09-13T17:44:50.336148+0000","last_unstale":"2024-09-13T17:44:50.337807+0000","last_undegraded":"2024-09-13T17:44:50.337807+0000","last_fullsized":"2024-09-13T17:44:50.337807+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:30:47.527759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.305244+0000","last_change":"2024-09-13T17:44:50.019493+0000","last_active":"2024-09-13T17:44:50.305244+0000","last_peered":"2024-09-13T17:44:50.305244+0000","last_clean":"2024-09-13T17:44:50.305244+0000","last_became_active":"2024-09-13T17:44:50.019073+0000","last_became_peered":"2024-09-13T17:44:50.019073+0000","last_unstale":"2024-09-13T17:44:50.305244+0000","last_undegraded":"2024-09-13T17:44:50.305244+0000","last_fullsized":"2024-09-13T17:44:50.305244+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:34:52.857441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.335847+0000","last_change":"2024-09-13T17:44:50.330795+0000","last_active":"2024-09-13T17:44:50.335847+0000","last_peered":"2024-09-13T17:44:50.335847+0000","last_clean":"2024-09-13T17:44:50.335847+0000","last_became_active":"2024-09-13T17:44:50.327102+0000","last_became_peered":"2024-09-13T17:44:50.327102+0000","last_unstale":"2024-09-13T17:44:50.335847+0000","last_undegraded":"2024-09-13T17:44:50.335847+0000","last_fullsized":"2024-09-13T17:44:50.335847+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:08:07.560502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-13T17:44:51.300603+0000","last_change":"2024-09-13T17:44:50.010422+0000","last_active":"2024-09-13T17:44:51.300603+0000","last_peered":"2024-09-13T17:44:51.300603+0000","last_clean":"2024-09-13T17:44:51.300603+0000","last_became_active":"2024-09-13T17:44:50.008092+0000","last_became_peered":"2024-09-13T17:44:50.008092+0000","last_unstale":"2024-09-13T17:44:51.300603+0000","last_undegraded":"2024-09-13T17:44:51.300603+0000","last_fullsized":"2024-09-13T17:44:51.300603+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:58:11.553887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.348572+0000","last_change":"2024-09-13T17:44:50.344579+0000","last_active":"2024-09-13T17:44:50.348572+0000","last_peered":"2024-09-13T17:44:50.348572+0000","last_clean":"2024-09-13T17:44:50.348572+0000","last_became_active":"2024-09-13T17:44:50.343095+0000","last_became_peered":"2024-09-13T17:44:50.343095+0000","last_unstale":"2024-09-13T17:44:50.348572+0000","last_undegraded":"2024-09-13T17:44:50.348572+0000","last_fullsized":"2024-09-13T17:44:50.348572+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:49.902372+0000","last_clean_scrub_stamp":"2024-09-13T17:44:49.902372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:35:55.586004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"55'2","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-13T17:44:50.336920+0000","last_change":"2024-09-13T17:44:49.949728+0000","last_active":"2024-09-13T17:44:50.336920+0000","last_peered":"2024-09-13T17:44:50.336920+0000","last_clean":"2024-09-13T17:44:50.336920+0000","last_became_active":"2024-09-13T17:44:49.949463+0000","last_became_peered":"2024-09-13T17:44:49.949463+0000","last_unstale":"2024-09-13T17:44:50.336920+0000","last_undegraded":"2024-09-13T17:44:50.336920+0000","last_fullsized":"2024-09-13T17:44:50.336920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:44:48.892190+0000","last_clean_scrub_stamp":"2024-09-13T17:44:48.892190+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:45:33.042083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"43'141","reported_seq":335,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-09-13T17:44:49.941310+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:44:49.941310+0000","last_peered":"2024-09-13T17:44:49.941310+0000","last_clean":"2024-09-13T17:44:49.941310+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:44:49.941310+0000","last_undegraded":"2024-09-13T17:44:49.941310+0000","last_fullsized":"2024-09-13T17:44:49.941310+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":217,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":490,"num_read_kb":915,"num_write":217,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141,"ondisk_log_size":141,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738406,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31684,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340156,"statfs":{"total":96636764160,"available":96604319744,"internally_reserved":0,"allocated":569344,"data_stored":211966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.442,"15min":0.439},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.50800000000000001,"15min":0.505},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.71999999999999997,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.34000000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44500000000000001,"15min":0.441},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.72199999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52300000000000002,"15min":0.52600000000000002},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.71799999999999997,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.40400000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738407,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32240,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339600,"statfs":{"total":96636764160,"available":96603750400,"internally_reserved":0,"allocated":1146880,"data_stored":798144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58999999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.31900000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:43:54 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46100000000000002,"15min":0.46200000000000002},"min":{"1min":0.27800000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.65000000000000002,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.46899999999999997,"15min":0.46300000000000002},"min":{"1min":0.26000000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.30199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738406,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1130496,"data_stored":796102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.29899999999999999,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.59599999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.502},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.495,"15min":0.495},"min":{"1min":0.29399999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.78800000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:43:52 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.28999999999999998,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.68700000000000006,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.28100000000000003,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70099999999999996,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56299999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:44:54.673 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:44:54.673 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:44:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae7a50640 1 Processor -- start 2024-09-13T17:44:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae7a50640 1 -- start start 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae7a50640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ae0135f10 0x7f2ae009e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae7a50640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ae009e060 con 0x7f2ae01362f0 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae7a50640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ae00a6740 con 0x7f2ae0135f10 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae57ec640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ae0135f10 0x7f2ae009e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae4feb640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2ae01362f0 0x7f2ae006dd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59258/0 (socket says 172.21.15.96:59258) 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae4feb640 1 -- 172.21.15.96:0/4022535807 learned_addr learned my addr 172.21.15.96:0/4022535807 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae57ec640 1 -- 172.21.15.96:0/4022535807 >> v1:172.21.15.96:6789/0 conn(0x7f2ae01362f0 legacy=0x7f2ae006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae57ec640 1 -- 172.21.15.96:0/4022535807 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae009edf0 con 0x7f2ae0135f10 2024-09-13T17:44:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ae57ec640 1 --2- 172.21.15.96:0/4022535807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae009e7b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009740 tx=0x7f2ad001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d28d7b416c073 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ad7fff640 1 -- 172.21.15.96:0/4022535807 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad00ab440 con 0x7f2ae0135f10 2024-09-13T17:44:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ad7fff640 1 -- 172.21.15.96:0/4022535807 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad00a7070 con 0x7f2ae0135f10 2024-09-13T17:44:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.831+0000 7f2ad7fff640 1 -- 172.21.15.96:0/4022535807 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad00ab6f0 con 0x7f2ae0135f10 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/4022535807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 msgr2=0x7f2ae009e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/4022535807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae009e7b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009740 tx=0x7f2ad001c420 comp rx=0 tx=0).stop 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/4022535807 shutdown_connections 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/4022535807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae009e7b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/4022535807 >> 172.21.15.96:0/4022535807 conn(0x7f2ae0098a60 msgr2=0x7f2ae009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/4022535807 shutdown_connections 2024-09-13T17:44:54.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/4022535807 wait complete. 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 Processor -- start 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- start start 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae01376c0 con 0x7f2ae0135f10 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae57ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae57ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40572/0 (socket says 172.21.15.96:40572) 2024-09-13T17:44:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae57ec640 1 -- 172.21.15.96:0/906300656 learned_addr learned my addr 172.21.15.96:0/906300656 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae57ec640 1 -- 172.21.15.96:0/906300656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad0009420 con 0x7f2ae0135f10 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae57ec640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2ad001a050 tx=0x7f2ad00191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad00abda0 con 0x7f2ae0135f10 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ad00bf410 con 0x7f2ae0135f10 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2ad00bf6d0 con 0x7f2ae0135f10 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ae00a52e0 con 0x7f2ae0135f10 2024-09-13T17:44:54.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ae00a6420 con 0x7f2ae0135f10 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f2ad00bf830 con 0x7f2ae0135f10 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ae00a27c0 con 0x7f2ae0135f10 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 0x7f2ac4037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f2ad00e2810 con 0x7f2ae0135f10 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.835+0000 7f2ae4feb640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 0x7f2ac4037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:54.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.839+0000 7f2ae4feb640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 0x7f2ac4037b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001610 tx=0x7f2adc007ad0 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:54.843+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ad00aa9f0 con 0x7f2ae0135f10 2024-09-13T17:44:55.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.059+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2ae00a30a0 con 0x7f2ae0135f10 2024-09-13T17:44:55.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.063+0000 7f2ad5ffb640 1 -- 172.21.15.96:0/906300656 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f2ad00b3390 con 0x7f2ae0135f10 2024-09-13T17:44:55.069 INFO:teuthology.orchestra.run.smithi096.stderr:updated caps for client.0 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 msgr2=0x7f2ac4037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 0x7f2ac4037b50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2adc001610 tx=0x7f2adc007ad0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 msgr2=0x7f2ae00a09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f2ad001a050 tx=0x7f2ad00191c0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 shutdown_connections 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f2ac4035690 0x7f2ac4037b50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 --2- 172.21.15.96:0/906300656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae0135f10 0x7f2ae00a09b0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.071 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 >> 172.21.15.96:0/906300656 conn(0x7f2ae0098a60 msgr2=0x7f2ae009a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 shutdown_connections 2024-09-13T17:44:55.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.067+0000 7f2ae7a50640 1 -- 172.21.15.96:0/906300656 wait complete. 2024-09-13T17:44:55.087 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:44:55.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f37669a0640 1 Processor -- start 2024-09-13T17:44:55.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f37669a0640 1 -- start start 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f37669a0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f376009dc20 0x7f37601351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f37669a0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376006e130 con 0x7f376006f6d0 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f37669a0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37600a6750 con 0x7f376009dc20 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f376009dc20 0x7f37601351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f376006f6d0 0x7f3760134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59270/0 (socket says 172.21.15.96:59270) 2024-09-13T17:44:55.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375ffff640 1 -- 172.21.15.96:0/2696444505 learned_addr learned my addr 172.21.15.96:0/2696444505 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:55.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375f7fe640 1 -- 172.21.15.96:0/2696444505 >> v1:172.21.15.96:6789/0 conn(0x7f376006f6d0 legacy=0x7f3760134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375effd640 1 -- 172.21.15.96:0/2696444505 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3753310131 0 0) 0x7f376006e130 con 0x7f376006f6d0 2024-09-13T17:44:55.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.199+0000 7f375f7fe640 1 -- 172.21.15.96:0/2696444505 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37600712e0 con 0x7f376009dc20 2024-09-13T17:44:55.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f375f7fe640 1 --2- 172.21.15.96:0/2696444505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376009dc20 0x7f37601351c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3754009740 tx=0x7f375401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6385429375986777 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f375effd640 1 -- 172.21.15.96:0/2696444505 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37540ab400 con 0x7f376009dc20 2024-09-13T17:44:55.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f375effd640 1 -- 172.21.15.96:0/2696444505 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37540a7050 con 0x7f376009dc20 2024-09-13T17:44:55.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f375effd640 1 -- 172.21.15.96:0/2696444505 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37540ab6b0 con 0x7f376009dc20 2024-09-13T17:44:55.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- 172.21.15.96:0/2696444505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376009dc20 msgr2=0x7f37601351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 --2- 172.21.15.96:0/2696444505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376009dc20 0x7f37601351c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3754009740 tx=0x7f375401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:55.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- 172.21.15.96:0/2696444505 shutdown_connections 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 --2- 172.21.15.96:0/2696444505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376009dc20 0x7f37601351c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- 172.21.15.96:0/2696444505 >> 172.21.15.96:0/2696444505 conn(0x7f3760098a60 msgr2=0x7f376009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- 172.21.15.96:0/2696444505 shutdown_connections 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- 172.21.15.96:0/2696444505 wait complete. 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 Processor -- start 2024-09-13T17:44:55.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.203+0000 7f37669a0640 1 -- start start 2024-09-13T17:44:55.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f37669a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f37669a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37600a12d0 con 0x7f376006f6d0 2024-09-13T17:44:55.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40578/0 (socket says 172.21.15.96:40578) 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375ffff640 1 -- 172.21.15.96:0/3537919299 learned_addr learned my addr 172.21.15.96:0/3537919299 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375ffff640 1 -- 172.21.15.96:0/3537919299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3754009420 con 0x7f376006f6d0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375ffff640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f375000c420 tx=0x7f375000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f37500074d0 con 0x7f376006f6d0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3750007630 con 0x7f376006f6d0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37600a0f50 con 0x7f376006f6d0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3750004960 con 0x7f376006f6d0 2024-09-13T17:44:55.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37600a1ed0 con 0x7f376006f6d0 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376006ead0 con 0x7f376006f6d0 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f3750004ac0 con 0x7f376006f6d0 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 0x7f374c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f3750047090 con 0x7f376006f6d0 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375f7fe640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 0x7f374c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.207+0000 7f375f7fe640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 0x7f374c037b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37540ad000 tx=0x7f3754008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.211+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f375001acd0 con 0x7f376006f6d0 2024-09-13T17:44:55.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.391+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37600a1580 con 0x7f376006f6d0 2024-09-13T17:44:55.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.391+0000 7f375cff9640 1 -- 172.21.15.96:0/3537919299 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f3750018070 con 0x7f376006f6d0 2024-09-13T17:44:55.397 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:44:55.397 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:44:55.398 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:44:55.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 msgr2=0x7f374c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 0x7f374c037b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f37540ad000 tx=0x7f3754008040 comp rx=0 tx=0).stop 2024-09-13T17:44:55.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 msgr2=0x7f376009e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f375000c420 tx=0x7f375000c8e0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 shutdown_connections 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f374c035690 0x7f374c037b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 --2- 172.21.15.96:0/3537919299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f376006f6d0 0x7f376009e1a0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 >> 172.21.15.96:0/3537919299 conn(0x7f3760098a60 msgr2=0x7f3760099a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 shutdown_connections 2024-09-13T17:44:55.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.395+0000 7f37669a0640 1 -- 172.21.15.96:0/3537919299 wait complete. 2024-09-13T17:44:55.616 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:44:50.294936+0000', 'modified': '2024-09-13T17:44:53.298866+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5163}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5163': {'gid': 5163, 'name': 'a', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6829/3293143360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 3293143360}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 3293143360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:44:55.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:44:55.617 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-13T17:44:55.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2f859640 1 Processor -- start 2024-09-13T17:44:55.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2f859640 1 -- start start 2024-09-13T17:44:55.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2f859640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fec2813af50 0x7fec2813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2f859640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec28138050 con 0x7fec2809d690 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2f859640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2809c410 con 0x7fec2813af50 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2cdf4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fec2813af50 0x7fec2813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2d5f5640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fec2809d690 0x7fec28138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59274/0 (socket says 172.21.15.96:59274) 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3466774812 learned_addr learned my addr 172.21.15.96:0/3466774812 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2cdf4640 1 -- 172.21.15.96:0/3466774812 >> v1:172.21.15.96:6789/0 conn(0x7fec2809d690 legacy=0x7fec28138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec2cdf4640 1 -- 172.21.15.96:0/3466774812 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2809cba0 con 0x7fec2813af50 2024-09-13T17:44:55.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.739+0000 7fec1ffff640 1 -- 172.21.15.96:0/3466774812 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3374565559 0 0) 0x7fec24001120 con 0x7fec2809d690 2024-09-13T17:44:55.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/3466774812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2813af50 0x7fec2813d340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fec18009740 tx=0x7fec1801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85a172863e83c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1ffff640 1 -- 172.21.15.96:0/3466774812 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec180a8840 con 0x7fec2813af50 2024-09-13T17:44:55.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1ffff640 1 -- 172.21.15.96:0/3466774812 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec180a89a0 con 0x7fec2813af50 2024-09-13T17:44:55.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1ffff640 1 -- 172.21.15.96:0/3466774812 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec180a8c60 con 0x7fec2813af50 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/3466774812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2813af50 msgr2=0x7fec2813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 --2- 172.21.15.96:0/3466774812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2813af50 0x7fec2813d340 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fec18009740 tx=0x7fec1801ec10 comp rx=0 tx=0).stop 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/3466774812 shutdown_connections 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 --2- 172.21.15.96:0/3466774812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2813af50 0x7fec2813d340 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/3466774812 >> 172.21.15.96:0/3466774812 conn(0x7fec2806bc60 msgr2=0x7fec2806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/3466774812 shutdown_connections 2024-09-13T17:44:55.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/3466774812 wait complete. 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 Processor -- start 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- start start 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec280a5e20 con 0x7fec2809d690 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40590/0 (socket says 172.21.15.96:40590) 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2d5f5640 1 -- 172.21.15.96:0/2717111869 learned_addr learned my addr 172.21.15.96:0/2717111869 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2d5f5640 1 -- 172.21.15.96:0/2717111869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009420 con 0x7fec2809d690 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2d5f5640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c390 tx=0x7fec2400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1dffb640 1 -- 172.21.15.96:0/2717111869 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec24007410 con 0x7fec2809d690 2024-09-13T17:44:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1dffb640 1 -- 172.21.15.96:0/2717111869 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24007590 con 0x7fec2809d690 2024-09-13T17:44:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec280a27e0 con 0x7fec2809d690 2024-09-13T17:44:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec1dffb640 1 -- 172.21.15.96:0/2717111869 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec240047f0 con 0x7fec2809d690 2024-09-13T17:44:55.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.743+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec280a4250 con 0x7fec2809d690 2024-09-13T17:44:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec1dffb640 1 -- 172.21.15.96:0/2717111869 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fec24004950 con 0x7fec2809d690 2024-09-13T17:44:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec1dffb640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 0x7fec0c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec1dffb640 1 -- 172.21.15.96:0/2717111869 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fec240476e0 con 0x7fec2809d690 2024-09-13T17:44:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 0x7fec0c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.752 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 0x7fec0c037b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec180b2000 tx=0x7fec18008040 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 Processor -- start 2024-09-13T17:44:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- start start 2024-09-13T17:44:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.753 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59f7c6e80 con 0x55b59f905710 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40606/0 (socket says 172.21.15.96:40606) 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3862032801 learned_addr learned my addr 172.21.15.96:0/3862032801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3862032801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b59f449890 con 0x55b59f905710 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec2d5f5640 1 --2- 172.21.15.96:0/3862032801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fec24014000 tx=0x7fec24047600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749d1bed9ab65b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec12ffd640 1 -- 172.21.15.96:0/3862032801 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec24045070 con 0x55b59f905710 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec12ffd640 1 -- 172.21.15.96:0/3862032801 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240382c0 con 0x55b59f905710 2024-09-13T17:44:55.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec12ffd640 1 -- 172.21.15.96:0/3862032801 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec240134f0 con 0x55b59f905710 2024-09-13T17:44:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- 172.21.15.96:0/3862032801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 msgr2=0x55b59f8fb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 --2- 172.21.15.96:0/3862032801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fec24014000 tx=0x7fec24047600 comp rx=0 tx=0).stop 2024-09-13T17:44:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- 172.21.15.96:0/3862032801 shutdown_connections 2024-09-13T17:44:55.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 --2- 172.21.15.96:0/3862032801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f8fb8f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- 172.21.15.96:0/3862032801 >> 172.21.15.96:0/3862032801 conn(0x55b59f8f6230 msgr2=0x55b59f8fb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- 172.21.15.96:0/3862032801 shutdown_connections 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- 172.21.15.96:0/3862032801 wait complete. 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 -- start start 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.747+0000 7fec329a2000 1 Processor -- start 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59f754af0 con 0x55b59f905710 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 10 client.0 _ll_register_callbacks cb 0x55b59f8eaae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40622/0 (socket says 172.21.15.96:40622) 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2d5f5640 1 -- 172.21.15.96:0/413268899 learned_addr learned my addr 172.21.15.96:0/413268899 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2d5f5640 1 -- 172.21.15.96:0/413268899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec280a4250 con 0x55b59f905710 2024-09-13T17:44:55.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2d5f5640 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fec24013bf0 tx=0x7fec24045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b0d046dfb2ff06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec24012bd0 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fec24012d30 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b59f70a5f0 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec24047050 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 10 client.5222 fetch_fsmap learned FSMap version 35 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55b59f712ab0 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b59f70a5f0 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) v1 ==== 2024+0+0 (secure 0 0 0) 0x7fec2401b060 con 0x55b59f905710 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 10 client.5222 fetch_fsmap finished waiting for FSMap version 35 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 10 client.5222 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 20 client.5222 populate_metadata read hostname 'smithi096' 2024-09-13T17:44:55.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f90b130 0x55b59f931b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 4 client.5222 mds_command: new command op to 5163 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 --> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b59f74a790 con 0x55b59f90b130 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f90b130 0x55b59f931b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f90b130 0x55b59f931b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597a7f8a8cae1fea server_cookie=f7841f2c28289864 in_seq=0 out_seq=0 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 10 client.5222 ms_handle_connect on v2:172.21.15.96:6828/3293143360 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 1 -- 172.21.15.96:0/413268899 <== mds.0 v2:172.21.15.96:6828/3293143360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55b59f74a790 con 0x55b59f90b130 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 10 client.5222 handle_command_reply: tid=0 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 client.5222 shutdown 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 msgr2=0x55b59f92b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fec24013bf0 tx=0x7fec24045000 comp rx=0 tx=0).stop 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 shutdown_connections 2024-09-13T17:44:55.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f90b130 0x55b59f931b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 --2- 172.21.15.96:0/413268899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f92b760 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec07fff640 0 client.5222 ms_handle_reset on v2:172.21.15.96:6828/3293143360 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 >> 172.21.15.96:0/413268899 conn(0x55b59f8f6230 msgr2=0x55b59f8fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 shutdown_connections 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 1 -- 172.21.15.96:0/413268899 wait complete. 2024-09-13T17:44:55.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.751+0000 7fec329a2000 20 client.5222 trim_cache size 0 max 16384 2024-09-13T17:44:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 Processor -- start 2024-09-13T17:44:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- start start 2024-09-13T17:44:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59f7520c0 con 0x55b59f905710 2024-09-13T17:44:55.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40632/0 (socket says 172.21.15.96:40632) 2024-09-13T17:44:55.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3874178692 learned_addr learned my addr 172.21.15.96:0/3874178692 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3874178692 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b59f6f8c00 con 0x55b59f905710 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec2d5f5640 1 --2- 172.21.15.96:0/3874178692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fec2400cda0 tx=0x7fec2403a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f88b84133cf39d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec137fe640 1 -- 172.21.15.96:0/3874178692 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec2400e040 con 0x55b59f905710 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec137fe640 1 -- 172.21.15.96:0/3874178692 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec24038190 con 0x55b59f905710 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec137fe640 1 -- 172.21.15.96:0/3874178692 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec2401b1b0 con 0x55b59f905710 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- 172.21.15.96:0/3874178692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 msgr2=0x55b59f904640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 --2- 172.21.15.96:0/3874178692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fec2400cda0 tx=0x7fec2403a050 comp rx=0 tx=0).stop 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- 172.21.15.96:0/3874178692 shutdown_connections 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 --2- 172.21.15.96:0/3874178692 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f904640 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- 172.21.15.96:0/3874178692 >> 172.21.15.96:0/3874178692 conn(0x55b59f8f6230 msgr2=0x55b59f903f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- 172.21.15.96:0/3874178692 shutdown_connections 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.767+0000 7fec329a2000 1 -- 172.21.15.96:0/3874178692 wait complete. 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- start start 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 Processor -- start 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b59f74d9b0 con 0x55b59f905710 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 10 client.0 _ll_register_callbacks cb 0x55b59f6fca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2d5f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40634/0 (socket says 172.21.15.96:40634) 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3790200878 learned_addr learned my addr 172.21.15.96:0/3790200878 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2d5f5640 1 -- 172.21.15.96:0/3790200878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2403c040 con 0x55b59f905710 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2d5f5640 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fec2401bb30 tx=0x7fec2401cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23ac9b85237aab56 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec2400e040 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fec24047260 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b59f71d460 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec2401a2b0 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 10 client.5224 fetch_fsmap learned FSMap version 35 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55b59f449cc0 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec2405d020 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) v1 ==== 2024+0+0 (secure 0 0 0) 0x7fec2401a4d0 con 0x55b59f905710 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 10 client.5224 fetch_fsmap finished waiting for FSMap version 35 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 10 client.5224 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 20 client.5224 populate_metadata read hostname 'smithi096' 2024-09-13T17:44:55.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f9635d0 0x55b59f9639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 4 client.5224 mds_command: new command op to 5163 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 --> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b59f7169c0 con 0x55b59f9635d0 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f9635d0 0x55b59f9639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 10 client.5224 ms_handle_connect on v2:172.21.15.96:6828/3293143360 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2cdf4640 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f9635d0 0x55b59f9639b0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c79449b29297024e server_cookie=1bb84de9d325a1ba in_seq=0 out_seq=0 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 1 -- 172.21.15.96:0/3790200878 <== mds.0 v2:172.21.15.96:6828/3293143360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b59f7169c0 con 0x55b59f9635d0 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 10 client.5224 handle_command_reply: tid=0 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 client.5224 shutdown 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 msgr2=0x55b59f963090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fec2401bb30 tx=0x7fec2401cfa0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 shutdown_connections 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:6828/3293143360,v1:172.21.15.96:6829/3293143360] conn(0x55b59f9635d0 0x55b59f9639b0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 --2- 172.21.15.96:0/3790200878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b59f905710 0x55b59f963090 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec07fff640 0 client.5224 ms_handle_reset on v2:172.21.15.96:6828/3293143360 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 >> 172.21.15.96:0/3790200878 conn(0x55b59f8f6230 msgr2=0x55b59f904750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 shutdown_connections 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 1 -- 172.21.15.96:0/3790200878 wait complete. 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec329a2000 20 client.5224 trim_cache size 0 max 16384 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "a54aa7b1-da4c-43a3-b94f-1607e77cbcc4", 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-13T17:44:55.782 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5163, 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 5.4803139569999999, 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 35, 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 56, 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 56, 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 10.277598300999999 2024-09-13T17:44:55.783 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 msgr2=0x7fec0c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 0x7fec0c037b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec180b2000 tx=0x7fec18008040 comp rx=0 tx=0).stop 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 msgr2=0x7fec280a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fec2400c390 tx=0x7fec2400c850 comp rx=0 tx=0).stop 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 shutdown_connections 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fec0c035690 0x7fec0c037b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 --2- 172.21.15.96:0/2717111869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec2809d690 0x7fec280a67f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 >> 172.21.15.96:0/2717111869 conn(0x7fec2806bc60 msgr2=0x7fec28139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 shutdown_connections 2024-09-13T17:44:55.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.771+0000 7fec2f859640 1 -- 172.21.15.96:0/2717111869 wait complete. 2024-09-13T17:44:55.794 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:44:55.890 INFO:tasks.ceph.mds.a.smithi096.stderr:2024-09-13T17:44:55.883+0000 7f207e835640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:44:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.891+0000 7f4692927640 1 Processor -- start 2024-09-13T17:44:55.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.891+0000 7f4692927640 1 -- start start 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.891+0000 7f4692927640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f468c0a1240 0x7f468c099a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.891+0000 7f4692927640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c0a0e10 con 0x7f468c0a1620 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.891+0000 7f4692927640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c0a0210 con 0x7f468c0a1240 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f468c0a1240 0x7f468c099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f468c0a1620 0x7f468c135b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59280/0 (socket says 172.21.15.96:59280) 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468b7fe640 1 -- 172.21.15.96:0/3797835312 learned_addr learned my addr 172.21.15.96:0/3797835312 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468bfff640 1 -- 172.21.15.96:0/3797835312 >> v1:172.21.15.96:6789/0 conn(0x7f468c0a1620 legacy=0x7f468c135b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468bfff640 1 -- 172.21.15.96:0/3797835312 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0a2480 con 0x7f468c0a1240 2024-09-13T17:44:55.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468affd640 1 -- 172.21.15.96:0/3797835312 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020749048 0 0) 0x7f468c0a0e10 con 0x7f468c0a1620 2024-09-13T17:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468bfff640 1 --2- 172.21.15.96:0/3797835312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c099a60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f467c009d90 tx=0x7f467c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52d5428f12e0579 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468affd640 1 -- 172.21.15.96:0/3797835312 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c0a5810 con 0x7f468c0a1240 2024-09-13T17:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468affd640 1 -- 172.21.15.96:0/3797835312 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0a5970 con 0x7f468c0a1240 2024-09-13T17:44:55.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f468affd640 1 -- 172.21.15.96:0/3797835312 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c0a5c30 con 0x7f468c0a1240 2024-09-13T17:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 -- 172.21.15.96:0/3797835312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 msgr2=0x7f468c099a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 --2- 172.21.15.96:0/3797835312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c099a60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f467c009d90 tx=0x7f467c01c880 comp rx=0 tx=0).stop 2024-09-13T17:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 -- 172.21.15.96:0/3797835312 shutdown_connections 2024-09-13T17:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 --2- 172.21.15.96:0/3797835312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c099a60 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:55.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 -- 172.21.15.96:0/3797835312 >> 172.21.15.96:0/3797835312 conn(0x7f468c0a6540 msgr2=0x7f468c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 -- 172.21.15.96:0/3797835312 shutdown_connections 2024-09-13T17:44:55.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 -- 172.21.15.96:0/3797835312 wait complete. 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.895+0000 7f4692927640 1 Processor -- start 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 -- start start 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c1388a0 con 0x7f468c0a1240 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40642/0 (socket says 172.21.15.96:40642) 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468bfff640 1 -- 172.21.15.96:0/1074108359 learned_addr learned my addr 172.21.15.96:0/1074108359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468bfff640 1 -- 172.21.15.96:0/1074108359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c009a70 con 0x7f468c0a1240 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468bfff640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f467c01a510 tx=0x7f467c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c0ad780 con 0x7f468c0a1240 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c0ad8e0 con 0x7f468c0a1240 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c137be0 con 0x7f468c0a1240 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f467c0adbc0 con 0x7f468c0a1240 2024-09-13T17:44:55.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c070fb0 con 0x7f468c0a1240 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f467c0b4050 con 0x7f468c0a1240 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c071a60 con 0x7f468c0a1240 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 0x7f466c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f467c0e2dd0 con 0x7f468c0a1240 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468b7fe640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 0x7f466c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.899+0000 7f468b7fe640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 0x7f466c037b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4674007fd0 tx=0x7f4674007980 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:55.909 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:55.903+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f467c0ca940 con 0x7f468c0a1240 2024-09-13T17:44:56.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.087+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f468c136bf0 con 0x7f468c0a1240 2024-09-13T17:44:56.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4688ff9640 1 -- 172.21.15.96:0/1074108359 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f467c0c7070 con 0x7f468c0a1240 2024-09-13T17:44:56.095 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 msgr2=0x7f466c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 0x7f466c037b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4674007fd0 tx=0x7f4674007980 comp rx=0 tx=0).stop 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 msgr2=0x7f468c138360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f467c01a510 tx=0x7f467c019680 comp rx=0 tx=0).stop 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 shutdown_connections 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f466c035690 0x7f466c037b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 --2- 172.21.15.96:0/1074108359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f468c0a1240 0x7f468c138360 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:56.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 >> 172.21.15.96:0/1074108359 conn(0x7f468c0a6540 msgr2=0x7f468c132db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:56.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 shutdown_connections 2024-09-13T17:44:56.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:56.091+0000 7f4692927640 1 -- 172.21.15.96:0/1074108359 wait complete. 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:44:56.113 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:56.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.114 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:44:56.119 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.120 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.121 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42386sec preferred_lft 42386sec 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:44:56.122 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:56.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.123 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:44:56.172 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:44:56.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.174 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:44:56.221 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:56.222 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:44:56.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:44:56.223 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:44:56.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.298 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:44:56.343 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:44:56.344 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:56.344 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:56.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.344 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-13T17:44:56.400 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:44:56.400 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:56.400 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:56.447 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:44:56.448 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:56.448 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:56.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:56.552 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:56.553 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:56.607 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:56.608 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:56.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.608 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:56.613 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:44:56.655 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:44:56.655 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:44:56.735 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:56.749 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:56.749 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:56.801 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:56.802 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:56.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:56.802 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:56.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-13T17:44:56.807+0000 7f5d97fc6580 -1 init, newargv = 0x55b91b7ae640 newargc=15 2024-09-13T17:44:56.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[42248]: starting ceph client 2024-09-13T17:44:56.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[42248]: starting fuse 2024-09-13T17:44:57.917 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:57.933 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:57.933 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:57.986 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:57.987 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:57.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:57.988 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:58.036 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:58.036 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T17:44:58.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:44:58.037 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:44:58.038 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:44:58.156 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.42248.asok 2024-09-13T17:44:58.340 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42248.asok 2024-09-13T17:44:58.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:58.340 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42248.asok status 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "42248", 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:44:58.499 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5230, 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 5230 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3239489439 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:58.500 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3239489439 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.5230 192.168.0.1:0/3239489439", 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/3239489439", 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 35, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 56, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:44:58.501 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:44:58.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:58.513 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:44:58.513 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:58.519 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:44:58.519 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:58.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:58.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:44:58.575 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:44:58.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.735+0000 7fbe62434640 1 Processor -- start 2024-09-13T17:44:58.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.735+0000 7fbe62434640 1 -- start start 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe5c135270 0x7fbe5c13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe5c070de0 con 0x7fbe5c134e90 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe5c071d60 con 0x7fbe5c135270 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe5c135270 0x7fbe5c13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbe5c134e90 0x7fbe5c071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59294/0 (socket says 172.21.15.96:59294) 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5bfff640 1 -- 172.21.15.96:0/3245310525 learned_addr learned my addr 172.21.15.96:0/3245310525 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:44:58.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5b7fe640 1 -- 172.21.15.96:0/3245310525 >> v1:172.21.15.96:6789/0 conn(0x7fbe5c134e90 legacy=0x7fbe5c071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:58.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5b7fe640 1 -- 172.21.15.96:0/3245310525 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c06f110 con 0x7fbe5c135270 2024-09-13T17:44:58.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5b7fe640 1 --2- 172.21.15.96:0/3245310525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c135270 0x7fbe5c13cfe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009740 tx=0x7fbe4401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b2f438a0b42fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:58.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5affd640 1 -- 172.21.15.96:0/3245310525 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe440ab530 con 0x7fbe5c135270 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5affd640 1 -- 172.21.15.96:0/3245310525 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440a7050 con 0x7fbe5c135270 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe5affd640 1 -- 172.21.15.96:0/3245310525 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe440ab7e0 con 0x7fbe5c135270 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- 172.21.15.96:0/3245310525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c135270 msgr2=0x7fbe5c13cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 --2- 172.21.15.96:0/3245310525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c135270 0x7fbe5c13cfe0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fbe44009740 tx=0x7fbe4401c390 comp rx=0 tx=0).stop 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- 172.21.15.96:0/3245310525 shutdown_connections 2024-09-13T17:44:58.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 --2- 172.21.15.96:0/3245310525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c135270 0x7fbe5c13cfe0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:58.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- 172.21.15.96:0/3245310525 >> 172.21.15.96:0/3245310525 conn(0x7fbe5c098a60 msgr2=0x7fbe5c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:58.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- 172.21.15.96:0/3245310525 shutdown_connections 2024-09-13T17:44:58.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.739+0000 7fbe62434640 1 -- 172.21.15.96:0/3245310525 wait complete. 2024-09-13T17:44:58.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 Processor -- start 2024-09-13T17:44:58.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 -- start start 2024-09-13T17:44:58.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:58.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c0a1550 con 0x7fbe5c134e90 2024-09-13T17:44:58.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:58.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40656/0 (socket says 172.21.15.96:40656) 2024-09-13T17:44:58.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5bfff640 1 -- 172.21.15.96:0/1237638168 learned_addr learned my addr 172.21.15.96:0/1237638168 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:44:58.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5bfff640 1 -- 172.21.15.96:0/1237638168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44009420 con 0x7fbe5c134e90 2024-09-13T17:44:58.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5bfff640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400cb20 tx=0x7fbe5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:58.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe5400abf0 con 0x7fbe5c134e90 2024-09-13T17:44:58.749 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5400ad50 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c0a41a0 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe54012400 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c0a2060 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c09a980 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7fbe54012560 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 0x7fbe3c037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fbe54047a40 con 0x7fbe5c134e90 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5b7fe640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 0x7fbe3c037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:44:58.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.743+0000 7fbe5b7fe640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 0x7fbe3c037b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ad000 tx=0x7fbe440a6000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:44:58.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.751+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe54007070 con 0x7fbe5c134e90 2024-09-13T17:44:58.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.942+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbe5c136cd0 con 0x7fbe5c134e90 2024-09-13T17:44:58.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.942+0000 7fbe58ff9640 1 -- 172.21.15.96:0/1237638168 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbe54017070 con 0x7fbe5c134e90 2024-09-13T17:44:58.950 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:44:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 msgr2=0x7fbe3c037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 0x7fbe3c037b50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe440ad000 tx=0x7fbe440a6000 comp rx=0 tx=0).stop 2024-09-13T17:44:58.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 msgr2=0x7fbe5c0a43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400cb20 tx=0x7fbe5400a060 comp rx=0 tx=0).stop 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 shutdown_connections 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7fbe3c035690 0x7fbe3c037b50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 --2- 172.21.15.96:0/1237638168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5c134e90 0x7fbe5c0a43f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 >> 172.21.15.96:0/1237638168 conn(0x7fbe5c098a60 msgr2=0x7fbe5c133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 shutdown_connections 2024-09-13T17:44:58.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:44:58.946+0000 7fbe62434640 1 -- 172.21.15.96:0/1237638168 wait complete. 2024-09-13T17:44:58.968 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:44:58.969 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:58.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:58.969 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:44:58.975 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:44:58.976 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:44:58.977 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:44:58.978 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:44:58.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:58.979 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:44:59.028 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:44:59.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.029 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:59.080 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:59.081 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:44:59.081 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:44:59.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:44:59.081 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:44:59.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.156 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:44:59.204 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:44:59.204 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:44:59.204 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:59.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.205 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:44:59.259 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:44:59.259 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:59.259 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:59.308 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:44:59.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:44:59.308 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:44:59.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:59.417 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:59.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:59.470 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:59.471 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:59.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.472 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:59.520 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:59.521 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:44:59.521 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:44:59.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:44:59.582 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:44:59.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:44:59.636 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:44:59.637 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:44:59.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:44:59.637 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:44:59.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:44:59.638+0000 7fdec55fc580 -1 init, newargv = 0x5611ab309640 newargc=15 2024-09-13T17:44:59.646 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[42355]: starting ceph client 2024-09-13T17:44:59.651 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:44:59.651 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[42355]: starting fuse 2024-09-13T17:45:00.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:00.768 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:00.769 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:00.824 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:00.825 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:00.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:00.826 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:00.871 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:45:00.871 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:45:00.872 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:45:00.872 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:45:00.873 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:45:00.874 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:45:00.992 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.42355.asok 2024-09-13T17:45:01.175 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42355.asok 2024-09-13T17:45:01.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:01.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42355.asok status 2024-09-13T17:45:01.335 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:01.335 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:45:01.335 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "42355", 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:45:01.336 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5236, 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 5236 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 4287097780 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:01.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 4287097780 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.5236 192.168.0.2:0/4287097780", 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/4287097780", 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 35, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 56, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:45:01.338 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:01.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:01.349 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:45:01.349 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:01.354 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:45:01.355 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:01.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:01.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:01.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-09-13T17:45:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.574+0000 7f3f52b6d640 1 Processor -- start 2024-09-13T17:45:01.582 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.574+0000 7f3f52b6d640 1 -- start start 2024-09-13T17:45:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f4c135270 0x7f3f4c13cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c070de0 con 0x7f3f4c134e90 2024-09-13T17:45:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c071d60 con 0x7f3f4c135270 2024-09-13T17:45:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f4c135270 0x7f3f4c13cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:01.583 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3f4c135270 0x7f3f4c13cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40670/0 (socket says 172.21.15.96:40670) 2024-09-13T17:45:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f50909640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3f4c134e90 0x7f3f4c071530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:59310/0 (socket says 172.21.15.96:59310) 2024-09-13T17:45:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 -- 172.21.15.96:0/1376507112 learned_addr learned my addr 172.21.15.96:0/1376507112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 -- 172.21.15.96:0/1376507112 >> v1:172.21.15.96:6789/0 conn(0x7f3f4c134e90 legacy=0x7f3f4c071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:01.584 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 -- 172.21.15.96:0/1376507112 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c06f110 con 0x7f3f4c135270 2024-09-13T17:45:01.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4bfff640 1 --2- 172.21.15.96:0/1376507112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c135270 0x7f3f4c13cfe0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3f400097f0 tx=0x7f3f4001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4831634c24450ac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:01.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4b7fe640 1 -- 172.21.15.96:0/1376507112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f400ab530 con 0x7f3f4c135270 2024-09-13T17:45:01.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4b7fe640 1 -- 172.21.15.96:0/1376507112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f400a7050 con 0x7f3f4c135270 2024-09-13T17:45:01.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f4b7fe640 1 -- 172.21.15.96:0/1376507112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f400ab7e0 con 0x7f3f4c135270 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1376507112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c135270 msgr2=0x7f3f4c13cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1376507112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c135270 0x7f3f4c13cfe0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f3f400097f0 tx=0x7f3f4001c450 comp rx=0 tx=0).stop 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1376507112 shutdown_connections 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1376507112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c135270 0x7f3f4c13cfe0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1376507112 >> 172.21.15.96:0/1376507112 conn(0x7f3f4c098a60 msgr2=0x7f3f4c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.578+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1376507112 shutdown_connections 2024-09-13T17:45:01.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1376507112 wait complete. 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 Processor -- start 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- start start 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0a51d0 con 0x7f3f4c134e90 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f50909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f50909640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40676/0 (socket says 172.21.15.96:40676) 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f50909640 1 -- 172.21.15.96:0/1214031754 learned_addr learned my addr 172.21.15.96:0/1214031754 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f50909640 1 -- 172.21.15.96:0/1214031754 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400094d0 con 0x7f3f4c134e90 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f50909640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c7f0 tx=0x7f3f4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:01.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f4400a610 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4400a770 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c133800 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f4400aa10 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c09a9a0 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c0a4d80 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) v1 ==== 43511+0+0 (secure 0 0 0) 0x7f3f4400ab70 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 0x7f3f38037b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f3f440472f0 con 0x7f3f4c134e90 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f4bfff640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 0x7f3f38037b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:01.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.582+0000 7f3f4bfff640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 0x7f3f38037b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f400ad000 tx=0x7f3f400a6000 comp rx=0 tx=0).ready entity=mgr.4986 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:01.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.590+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3f4c0a4d80 con 0x7f3f4c134e90 2024-09-13T17:45:01.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:01.786+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) v1 -- 0x7f3f4c0a4d80 con 0x7f3f4c134e90 2024-09-13T17:45:02.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.662+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 26) v1 ==== 43520+0+0 (secure 0 0 0) 0x7f3f440073f0 con 0x7f3f4c134e90 2024-09-13T17:45:02.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.662+0000 7f3f497fa640 1 -- 172.21.15.96:0/1214031754 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v26) v1 ==== 84+0+0 (secure 0 0 0) 0x7f3f4400ae20 con 0x7f3f4c134e90 2024-09-13T17:45:02.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 msgr2=0x7f3f38037b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 0x7f3f38037b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f400ad000 tx=0x7f3f400a6000 comp rx=0 tx=0).stop 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 msgr2=0x7f3f4c0a2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400c7f0 tx=0x7f3f4400ccb0 comp rx=0 tx=0).stop 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 shutdown_connections 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:6824/3506364525,v1:172.21.15.96:6825/3506364525] conn(0x7f3f38035690 0x7f3f38037b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 --2- 172.21.15.96:0/1214031754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f4c134e90 0x7f3f4c0a2910 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 >> 172.21.15.96:0/1214031754 conn(0x7f3f4c098a60 msgr2=0x7f3f4c133dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 shutdown_connections 2024-09-13T17:45:02.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:02.666+0000 7f3f52b6d640 1 -- 172.21.15.96:0/1214031754 wait complete. 2024-09-13T17:45:02.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:02.692 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --id=admin --selftest) 2024-09-13T17:45:02.797 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:02.790+0000 7f78784d6180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:45:03.160 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.154+0000 7f78784d6180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:45:03.287 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.282+0000 7f78784d6180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:45:03.702 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.694+0000 7f78784d6180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:03.752 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.746+0000 7f78784d6180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:45:03.800 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.794+0000 7f78784d6180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:45:03.874 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.866+0000 7f78784d6180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:45:03.923 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.918+0000 7f78784d6180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:45:03.995 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:03.990+0000 7f78784d6180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:45:04.044 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.038+0000 7f78784d6180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:45:04.183 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.178+0000 7f78784d6180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:45:04.233 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.226+0000 7f78784d6180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:45:04.450 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:45:04.451 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:45:04.451 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:45:04.458 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.450+0000 7f78784d6180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:45:04.589 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.582+0000 7f78784d6180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:04.697 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.690+0000 7f78784d6180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:04.757 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.750+0000 7f78784d6180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:45:04.888 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:04.882+0000 7f78784d6180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:05.363 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.358+0000 7f78784d6180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:45:05.436 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.430+0000 7f78784d6180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:45:05.577 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.570+0000 7f78784d6180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:45:05.649 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.642+0000 7f78784d6180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:45:05.931 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.926+0000 7f78784d6180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:45:05.985 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:05.978+0000 7f78784d6180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f780224f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f780224f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f780224f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f780224f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f780224f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f77ff249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f77ff249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f77ff249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f77ff249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:06.076 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:06.070+0000 7f77ff249640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:07.028 INFO:teuthology.orchestra.run.smithi096.stdout:selftest ok 2024-09-13T17:45:07.038 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-09-13T17:45:07.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e92e2640 1 Processor -- start 2024-09-13T17:45:07.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e92e2640 1 -- start start 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e92e2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6e4137d60 0x7fe6e40a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e92e2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e40a00b0 con 0x7fe6e4138140 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e92e2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e40a12d0 con 0x7fe6e4137d60 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.154+0000 7fe6e2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6e4137d60 0x7fe6e40a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e2ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe6e4137d60 0x7fe6e40a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49070/0 (socket says 172.21.15.96:49070) 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e2ffd640 1 -- 172.21.15.96:0/2115220489 learned_addr learned my addr 172.21.15.96:0/2115220489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e2ffd640 1 -- 172.21.15.96:0/2115220489 >> v1:172.21.15.96:6789/0 conn(0x7fe6e4138140 legacy=0x7fe6e4138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:07.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e2ffd640 1 -- 172.21.15.96:0/2115220489 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40a0350 con 0x7fe6e4137d60 2024-09-13T17:45:07.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e2ffd640 1 --2- 172.21.15.96:0/2115220489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a0d90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40097f0 tx=0x7fe6d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed6299521320499 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:07.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e1ffb640 1 -- 172.21.15.96:0/2115220489 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d40ab530 con 0x7fe6e4137d60 2024-09-13T17:45:07.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e1ffb640 1 -- 172.21.15.96:0/2115220489 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d40a7050 con 0x7fe6e4137d60 2024-09-13T17:45:07.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e1ffb640 1 -- 172.21.15.96:0/2115220489 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d40ab7e0 con 0x7fe6e4137d60 2024-09-13T17:45:07.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 -- 172.21.15.96:0/2115220489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 msgr2=0x7fe6e40a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/2115220489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a0d90 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe6d40097f0 tx=0x7fe6d401c450 comp rx=0 tx=0).stop 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 -- 172.21.15.96:0/2115220489 shutdown_connections 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/2115220489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a0d90 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 -- 172.21.15.96:0/2115220489 >> 172.21.15.96:0/2115220489 conn(0x7fe6e406ef80 msgr2=0x7fe6e40713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 -- 172.21.15.96:0/2115220489 shutdown_connections 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.158+0000 7fe6e92e2640 1 -- 172.21.15.96:0/2115220489 wait complete. 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 Processor -- start 2024-09-13T17:45:07.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 -- start start 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e40a2760 con 0x7fe6e4137d60 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49082/0 (socket says 172.21.15.96:49082) 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e2ffd640 1 -- 172.21.15.96:0/3276858678 learned_addr learned my addr 172.21.15.96:0/3276858678 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e2ffd640 1 -- 172.21.15.96:0/3276858678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d40094d0 con 0x7fe6e4137d60 2024-09-13T17:45:07.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e2ffd640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe6d401a080 tx=0x7fe6d401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d40b6630 con 0x7fe6e4137d60 2024-09-13T17:45:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6d40b6790 con 0x7fe6e4137d60 2024-09-13T17:45:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e40a2310 con 0x7fe6e4137d60 2024-09-13T17:45:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe6d40b6a50 con 0x7fe6e4137d60 2024-09-13T17:45:07.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e40a3130 con 0x7fe6e4137d60 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 28) v1 ==== 43154+0+0 (secure 0 0 0) 0x7fe6d40b2040 con 0x7fe6e4137d60 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e4136600 con 0x7fe6e4137d60 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 0x7fe6d00377b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fe6d40e1d10 con 0x7fe6e4137d60 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.162+0000 7fe6e27fc640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 0x7fe6d00377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:07.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.166+0000 7fe6e27fc640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 0x7fe6d00377b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f70 tx=0x7fe6d8007a40 comp rx=0 tx=0).ready entity=mgr.5242 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:07.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.170+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fe6d40bd950 con 0x7fe6e4137d60 2024-09-13T17:45:07.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:07.366+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) v1 -- 0x7fe6e40a2530 con 0x7fe6e4137d60 2024-09-13T17:45:08.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.070+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fe6d40b43f0 con 0x7fe6e4137d60 2024-09-13T17:45:08.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6cb7fe640 1 -- 172.21.15.96:0/3276858678 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v29) v1 ==== 85+0+0 (secure 0 0 0) 0x7fe6d40b37b0 con 0x7fe6e4137d60 2024-09-13T17:45:08.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 msgr2=0x7fe6d00377b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 0x7fe6d00377b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8000f70 tx=0x7fe6d8007a40 comp rx=0 tx=0).stop 2024-09-13T17:45:08.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 msgr2=0x7fe6e40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fe6d401a080 tx=0x7fe6d401ac90 comp rx=0 tx=0).stop 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 shutdown_connections 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fe6d00352f0 0x7fe6d00377b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 --2- 172.21.15.96:0/3276858678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e4137d60 0x7fe6e40a2a60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 >> 172.21.15.96:0/3276858678 conn(0x7fe6e406ef80 msgr2=0x7fe6e40706c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 shutdown_connections 2024-09-13T17:45:08.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.074+0000 7fe6e92e2640 1 -- 172.21.15.96:0/3276858678 wait complete. 2024-09-13T17:45:08.101 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:45:08.102 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:45:08.193 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:45:08.193 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:08.194 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:08.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:08.201 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:08.194+0000 7f30adca0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:45:08.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c3744c640 1 Processor -- start 2024-09-13T17:45:08.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c3744c640 1 -- start start 2024-09-13T17:45:08.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c3744c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7c3009c180 0x7f7c3013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c3744c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c3009d040 con 0x7f7c30132a10 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c3744c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c30133500 con 0x7f7c3009c180 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c351e8640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7c30132a10 0x7f7c30132df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57494/0 (socket says 172.21.15.96:57494) 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c351e8640 1 -- 172.21.15.96:0/433990272 learned_addr learned my addr 172.21.15.96:0/433990272 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c349e7640 1 --2- 172.21.15.96:0/433990272 >> v2:172.21.15.96:3300/0 conn(0x7f7c3009c180 0x7f7c3013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c27fff640 1 -- 172.21.15.96:0/433990272 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870898931 0 0) 0x7f7c3009d040 con 0x7f7c30132a10 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c27fff640 1 -- 172.21.15.96:0/433990272 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c1c003530 con 0x7f7c30132a10 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c349e7640 1 -- 172.21.15.96:0/433990272 >> v1:172.21.15.96:6789/0 conn(0x7f7c30132a10 legacy=0x7f7c30132df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.302+0000 7f7c349e7640 1 -- 172.21.15.96:0/433990272 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3009f080 con 0x7f7c3009c180 2024-09-13T17:45:08.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c349e7640 1 --2- 172.21.15.96:0/433990272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c3013ae50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009740 tx=0x7f7c2001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafc8d8e4974aa47 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c27fff640 1 -- 172.21.15.96:0/433990272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c200ab400 con 0x7f7c3009c180 2024-09-13T17:45:08.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c27fff640 1 -- 172.21.15.96:0/433990272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c200a7050 con 0x7f7c3009c180 2024-09-13T17:45:08.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c27fff640 1 -- 172.21.15.96:0/433990272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c200ab690 con 0x7f7c3009c180 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c3744c640 1 -- 172.21.15.96:0/433990272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 msgr2=0x7f7c3013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c3744c640 1 --2- 172.21.15.96:0/433990272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c3013ae50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7c20009740 tx=0x7f7c2001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c3744c640 1 -- 172.21.15.96:0/433990272 shutdown_connections 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c3744c640 1 --2- 172.21.15.96:0/433990272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c3013ae50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.306+0000 7f7c3744c640 1 -- 172.21.15.96:0/433990272 >> 172.21.15.96:0/433990272 conn(0x7f7c3006e020 msgr2=0x7f7c30070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- 172.21.15.96:0/433990272 shutdown_connections 2024-09-13T17:45:08.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- 172.21.15.96:0/433990272 wait complete. 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 Processor -- start 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- start start 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300a19c0 con 0x7f7c3009c180 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c351e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c351e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49124/0 (socket says 172.21.15.96:49124) 2024-09-13T17:45:08.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c351e8640 1 -- 172.21.15.96:0/1081520464 learned_addr learned my addr 172.21.15.96:0/1081520464 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c351e8640 1 -- 172.21.15.96:0/1081520464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c20009420 con 0x7f7c3009c180 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c351e8640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3d0 tx=0x7f7c2c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c2c005ce0 con 0x7f7c3009c180 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c005e40 con 0x7f7c3009c180 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c300a0d30 con 0x7f7c3009c180 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7c2c007220 con 0x7f7c3009c180 2024-09-13T17:45:08.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3009fd30 con 0x7f7c3009c180 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f7c2c007440 con 0x7f7c3009c180 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3009ff80 con 0x7f7c3009c180 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 0x7f7c14037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f7c2c047040 con 0x7f7c3009c180 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c349e7640 1 -- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 msgr2=0x7f7c14037950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:08.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.310+0000 7f7c349e7640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 0x7f7c14037950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:08.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.314+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7c3009ff80 con 0x7f7c3009c180 2024-09-13T17:45:08.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.474+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c3009ff80 con 0x7f7c3009c180 2024-09-13T17:45:08.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.474+0000 7f7c25ffb640 1 -- 172.21.15.96:0/1081520464 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+11118 (secure 0 0 0) 0x7f7c3009ff80 con 0x7f7c3009c180 2024-09-13T17:45:08.482 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:08.482 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":57,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:06.004320+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:08.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 msgr2=0x7f7c14037950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:08.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 0x7f7c14037950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 msgr2=0x7f7c300a1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00c3d0 tx=0x7f7c2c00c890 comp rx=0 tx=0).stop 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 shutdown_connections 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f7c14035490 0x7f7c14037950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 --2- 172.21.15.96:0/1081520464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c3009c180 0x7f7c300a1480 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 >> 172.21.15.96:0/1081520464 conn(0x7f7c3006e020 msgr2=0x7f7c3012f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 shutdown_connections 2024-09-13T17:45:08.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.478+0000 7f7c3744c640 1 -- 172.21.15.96:0/1081520464 wait complete. 2024-09-13T17:45:08.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:44:48.116392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.0299999713897705, 'score_stable': 1.0299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.0299999713897705, 'raw_score_stable': 1.0299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:44:49.284535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:08.501 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:08.585 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:08.574+0000 7f30adca0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:45:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.602+0000 7f558c784640 1 Processor -- start 2024-09-13T17:45:08.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.602+0000 7f558c784640 1 -- start start 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558c784640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558413af50 0x7f558413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558c784640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5584138050 con 0x7f558409d690 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558c784640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558409c410 con 0x7f558413af50 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558413af50 0x7f558413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f558413af50 0x7f558413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49126/0 (socket says 172.21.15.96:49126) 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558a520640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f558409d690 0x7f5584138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57510/0 (socket says 172.21.15.96:57510) 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 -- 172.21.15.96:0/475092264 learned_addr learned my addr 172.21.15.96:0/475092264 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 -- 172.21.15.96:0/475092264 >> v1:172.21.15.96:6789/0 conn(0x7f558409d690 legacy=0x7f5584138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 -- 172.21.15.96:0/475092264 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558409cba0 con 0x7f558413af50 2024-09-13T17:45:08.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558951e640 1 -- 172.21.15.96:0/475092264 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956645798 0 0) 0x7f5580001120 con 0x7f558409d690 2024-09-13T17:45:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f5589d1f640 1 --2- 172.21.15.96:0/475092264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558413af50 0x7f558413d340 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f55740097f0 tx=0x7f557401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735143f16b43f6d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558951e640 1 -- 172.21.15.96:0/475092264 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55740a8900 con 0x7f558413af50 2024-09-13T17:45:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558951e640 1 -- 172.21.15.96:0/475092264 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55740a8a60 con 0x7f558413af50 2024-09-13T17:45:08.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558951e640 1 -- 172.21.15.96:0/475092264 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55740a8d20 con 0x7f558413af50 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558c784640 1 -- 172.21.15.96:0/475092264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558413af50 msgr2=0x7f558413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.606+0000 7f558c784640 1 --2- 172.21.15.96:0/475092264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558413af50 0x7f558413d340 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f55740097f0 tx=0x7f557401ecd0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/475092264 shutdown_connections 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 --2- 172.21.15.96:0/475092264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558413af50 0x7f558413d340 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/475092264 >> 172.21.15.96:0/475092264 conn(0x7f558406bc60 msgr2=0x7f558406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/475092264 shutdown_connections 2024-09-13T17:45:08.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/475092264 wait complete. 2024-09-13T17:45:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 Processor -- start 2024-09-13T17:45:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- start start 2024-09-13T17:45:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55840a5c90 con 0x7f558409d690 2024-09-13T17:45:08.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558a520640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558a520640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49134/0 (socket says 172.21.15.96:49134) 2024-09-13T17:45:08.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558a520640 1 -- 172.21.15.96:0/3604456499 learned_addr learned my addr 172.21.15.96:0/3604456499 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558a520640 1 -- 172.21.15.96:0/3604456499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740094d0 con 0x7f558409d690 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558a520640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f558000c390 tx=0x7f558000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5580007410 con 0x7f558409d690 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5580007590 con 0x7f558409d690 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55840a6f20 con 0x7f558409d690 2024-09-13T17:45:08.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f55800047f0 con 0x7f558409d690 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5584137400 con 0x7f558409d690 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5580004950 con 0x7f558409d690 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558409bc00 con 0x7f558409d690 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 0x7f55680379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f5589d1f640 1 -- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 msgr2=0x7f55680379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f55800474e0 con 0x7f558409d690 2024-09-13T17:45:08.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.610+0000 7f5589d1f640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 0x7f55680379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:08.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.618+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f558409bc00 con 0x7f558409d690 2024-09-13T17:45:08.715 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:08.710+0000 7f30adca0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:45:08.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.766+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5584138050 con 0x7f558409d690 2024-09-13T17:45:08.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.766+0000 7f557affd640 1 -- 172.21.15.96:0/3604456499 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5584138050 con 0x7f558409d690 2024-09-13T17:45:08.774 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 msgr2=0x7f55680379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 0x7f55680379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 msgr2=0x7f55840a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f558000c390 tx=0x7f558000c850 comp rx=0 tx=0).stop 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 shutdown_connections 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f55680354e0 0x7f55680379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 --2- 172.21.15.96:0/3604456499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f558409d690 0x7f55840a69e0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 >> 172.21.15.96:0/3604456499 conn(0x7f558406bc60 msgr2=0x7f558409f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 shutdown_connections 2024-09-13T17:45:08.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.770+0000 7f558c784640 1 -- 172.21.15.96:0/3604456499 wait complete. 2024-09-13T17:45:08.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.902+0000 7fd57f833640 1 Processor -- start 2024-09-13T17:45:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.902+0000 7fd57f833640 1 -- start start 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.902+0000 7fd57f833640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd57813af50 0x7fd57813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.902+0000 7fd57f833640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd578138050 con 0x7fd57809d690 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.902+0000 7fd57f833640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd57809c410 con 0x7fd57813af50 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57cdce640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd57813af50 0x7fd57813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57d5cf640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd57809d690 0x7fd578138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57524/0 (socket says 172.21.15.96:57524) 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57d5cf640 1 -- 172.21.15.96:0/2492740473 learned_addr learned my addr 172.21.15.96:0/2492740473 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57cdce640 1 -- 172.21.15.96:0/2492740473 >> v1:172.21.15.96:6789/0 conn(0x7fd57809d690 legacy=0x7fd578138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57cdce640 1 -- 172.21.15.96:0/2492740473 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57809cba0 con 0x7fd57813af50 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57cdce640 1 --2- 172.21.15.96:0/2492740473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57813af50 0x7fd57813d340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd568009740 tx=0x7fd56801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852cbd6be3df17b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd56ffff640 1 -- 172.21.15.96:0/2492740473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5680a8840 con 0x7fd57813af50 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd56ffff640 1 -- 172.21.15.96:0/2492740473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5680a89a0 con 0x7fd57813af50 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd56ffff640 1 -- 172.21.15.96:0/2492740473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5680a8c60 con 0x7fd57813af50 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 -- 172.21.15.96:0/2492740473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57813af50 msgr2=0x7fd57813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 --2- 172.21.15.96:0/2492740473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57813af50 0x7fd57813d340 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd568009740 tx=0x7fd56801ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 -- 172.21.15.96:0/2492740473 shutdown_connections 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 --2- 172.21.15.96:0/2492740473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57813af50 0x7fd57813d340 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 -- 172.21.15.96:0/2492740473 >> 172.21.15.96:0/2492740473 conn(0x7fd57806bc60 msgr2=0x7fd57806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:08.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 -- 172.21.15.96:0/2492740473 shutdown_connections 2024-09-13T17:45:08.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.906+0000 7fd57f833640 1 -- 172.21.15.96:0/2492740473 wait complete. 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 Processor -- start 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 -- start start 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5780a4f90 con 0x7fd57809d690 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57d5cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57d5cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49140/0 (socket says 172.21.15.96:49140) 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57d5cf640 1 -- 172.21.15.96:0/2014567267 learned_addr learned my addr 172.21.15.96:0/2014567267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57d5cf640 1 -- 172.21.15.96:0/2014567267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009420 con 0x7fd57809d690 2024-09-13T17:45:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57d5cf640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c380 tx=0x7fd57400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd574005c20 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574005d80 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5780a2c20 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd574007200 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5781373b0 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fd574007360 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5780a3f70 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 0x7fd55c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57cdce640 1 -- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 msgr2=0x7fd55c0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fd574046ae0 con 0x7fd57809d690 2024-09-13T17:45:08.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.910+0000 7fd57cdce640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 0x7fd55c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:08.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:08.914+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd5780a3f70 con 0x7fd57809d690 2024-09-13T17:45:09.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.066+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd578138050 con 0x7fd57809d690 2024-09-13T17:45:09.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd56dffb640 1 -- 172.21.15.96:0/2014567267 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd578138050 con 0x7fd57809d690 2024-09-13T17:45:09.076 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:09.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 msgr2=0x7fd55c0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 0x7fd55c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 msgr2=0x7fd5781c9820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fd57400c380 tx=0x7fd57400c840 comp rx=0 tx=0).stop 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 shutdown_connections 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd55c0354e0 0x7fd55c0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 --2- 172.21.15.96:0/2014567267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd57809d690 0x7fd5781c9820 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 >> 172.21.15.96:0/2014567267 conn(0x7fd57806bc60 msgr2=0x7fd578131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 shutdown_connections 2024-09-13T17:45:09.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.070+0000 7fd57f833640 1 -- 172.21.15.96:0/2014567267 wait complete. 2024-09-13T17:45:09.095 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:09.143 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.138+0000 7f30adca0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:09.195 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.190+0000 7f30adca0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:45:09.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.210+0000 7fb3056a8640 1 Processor -- start 2024-09-13T17:45:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.210+0000 7fb3056a8640 1 -- start start 2024-09-13T17:45:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb3056a8640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb30006f6c0 0x7fb3000a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb3056a8640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30006eee0 con 0x7fb30006faa0 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb3056a8640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb300070090 con 0x7fb30006f6c0 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2feffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb30006f6c0 0x7fb3000a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fe7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb30006faa0 0x7fb3000a19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57534/0 (socket says 172.21.15.96:57534) 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fe7fc640 1 -- 172.21.15.96:0/1655711886 learned_addr learned my addr 172.21.15.96:0/1655711886 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2feffd640 1 -- 172.21.15.96:0/1655711886 >> v1:172.21.15.96:6789/0 conn(0x7fb30006faa0 legacy=0x7fb3000a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2feffd640 1 -- 172.21.15.96:0/1655711886 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3000a3040 con 0x7fb30006f6c0 2024-09-13T17:45:09.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fdffb640 1 -- 172.21.15.96:0/1655711886 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631522508 0 0) 0x7fb30006eee0 con 0x7fb30006faa0 2024-09-13T17:45:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2feffd640 1 --2- 172.21.15.96:0/1655711886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3000a1470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009dc0 tx=0x7fb2f001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda0127ef9bfc988 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fdffb640 1 -- 172.21.15.96:0/1655711886 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2f00a5790 con 0x7fb30006f6c0 2024-09-13T17:45:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fdffb640 1 -- 172.21.15.96:0/1655711886 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00a58f0 con 0x7fb30006f6c0 2024-09-13T17:45:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.214+0000 7fb2fdffb640 1 -- 172.21.15.96:0/1655711886 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2f00a5bb0 con 0x7fb30006f6c0 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/1655711886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 msgr2=0x7fb3000a1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 --2- 172.21.15.96:0/1655711886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3000a1470 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0009dc0 tx=0x7fb2f001c8c0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/1655711886 shutdown_connections 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 --2- 172.21.15.96:0/1655711886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3000a1470 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/1655711886 >> 172.21.15.96:0/1655711886 conn(0x7fb30006a150 msgr2=0x7fb30006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/1655711886 shutdown_connections 2024-09-13T17:45:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/1655711886 wait complete. 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 Processor -- start 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- start start 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb30009f530 con 0x7fb30006f6c0 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2feffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2feffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49160/0 (socket says 172.21.15.96:49160) 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2feffd640 1 -- 172.21.15.96:0/3231322112 learned_addr learned my addr 172.21.15.96:0/3231322112 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2feffd640 1 -- 172.21.15.96:0/3231322112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f0009a70 con 0x7fb30006f6c0 2024-09-13T17:45:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2feffd640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f001a4f0 tx=0x7fb2f0019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2f00ad800 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00ad960 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb2f00adc00 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb300138820 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb300136860 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3000a1470 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fb2f00b4050 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 0x7fb2ec0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fb2f00e2d70 con 0x7fb30006f6c0 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2fe7fc640 1 -- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 msgr2=0x7fb2ec0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.218+0000 7fb2fe7fc640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 0x7fb2ec0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:09.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.230+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb2f00ca940 con 0x7fb30006f6c0 2024-09-13T17:45:09.249 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.242+0000 7f30adca0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:45:09.326 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.318+0000 7f30adca0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:45:09.375 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.370+0000 7f30adca0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:45:09.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb3001219c0 con 0x7fb30006f6c0 2024-09-13T17:45:09.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb2e77fe640 1 -- 172.21.15.96:0/3231322112 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2f00c7070 con 0x7fb30006f6c0 2024-09-13T17:45:09.392 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:09.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 msgr2=0x7fb2ec0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 0x7fb2ec0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 msgr2=0x7fb3001cf430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f001a4f0 tx=0x7fb2f0019660 comp rx=0 tx=0).stop 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 shutdown_connections 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2ec0354e0 0x7fb2ec0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 --2- 172.21.15.96:0/3231322112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb30006f6c0 0x7fb3001cf430 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 >> 172.21.15.96:0/3231322112 conn(0x7fb30006a150 msgr2=0x7fb3000a5640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 shutdown_connections 2024-09-13T17:45:09.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.386+0000 7fb3056a8640 1 -- 172.21.15.96:0/3231322112 wait complete. 2024-09-13T17:45:09.411 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:45:09.451 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.446+0000 7f30adca0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:45:09.501 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.494+0000 7f30adca0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:45:09.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f732700e640 1 Processor -- start 2024-09-13T17:45:09.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f732700e640 1 -- start start 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f732700e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f732013af50 0x7f732013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f732700e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7320138050 con 0x7f732009d690 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f732700e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732009c410 con 0x7f732013af50 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f7324daa640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f732009d690 0x7f7320138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57548/0 (socket says 172.21.15.96:57548) 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f731ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f732013af50 0x7f732013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:09.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f7324daa640 1 -- 172.21.15.96:0/1772028871 learned_addr learned my addr 172.21.15.96:0/1772028871 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:09.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f731ffff640 1 -- 172.21.15.96:0/1772028871 >> v1:172.21.15.96:6789/0 conn(0x7f732009d690 legacy=0x7f7320138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f731ffff640 1 -- 172.21.15.96:0/1772028871 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732009cba0 con 0x7f732013af50 2024-09-13T17:45:09.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.518+0000 7f731f7fe640 1 -- 172.21.15.96:0/1772028871 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263870318 0 0) 0x7f7310001120 con 0x7f732009d690 2024-09-13T17:45:09.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f731ffff640 1 --2- 172.21.15.96:0/1772028871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732013af50 0x7f732013d340 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7314009740 tx=0x7f731401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65012adccb78a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:09.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f731f7fe640 1 -- 172.21.15.96:0/1772028871 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73140a8840 con 0x7f732013af50 2024-09-13T17:45:09.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f731f7fe640 1 -- 172.21.15.96:0/1772028871 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140a89a0 con 0x7f732013af50 2024-09-13T17:45:09.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f731f7fe640 1 -- 172.21.15.96:0/1772028871 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73140a8c60 con 0x7f732013af50 2024-09-13T17:45:09.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- 172.21.15.96:0/1772028871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732013af50 msgr2=0x7f732013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 --2- 172.21.15.96:0/1772028871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732013af50 0x7f732013d340 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f7314009740 tx=0x7f731401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:09.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- 172.21.15.96:0/1772028871 shutdown_connections 2024-09-13T17:45:09.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 --2- 172.21.15.96:0/1772028871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732013af50 0x7f732013d340 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- 172.21.15.96:0/1772028871 >> 172.21.15.96:0/1772028871 conn(0x7f732006bc60 msgr2=0x7f732006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:09.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- 172.21.15.96:0/1772028871 shutdown_connections 2024-09-13T17:45:09.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- 172.21.15.96:0/1772028871 wait complete. 2024-09-13T17:45:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 Processor -- start 2024-09-13T17:45:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.522+0000 7f732700e640 1 -- start start 2024-09-13T17:45:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f732700e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f732700e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73200a6f20 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f7324daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f7324daa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49188/0 (socket says 172.21.15.96:49188) 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f7324daa640 1 -- 172.21.15.96:0/1024830833 learned_addr learned my addr 172.21.15.96:0/1024830833 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f7324daa640 1 -- 172.21.15.96:0/1024830833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009420 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f7324daa640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f731000c390 tx=0x7f731000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7310007410 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7310007590 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73200a4cb0 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f73100047f0 con 0x7f732009d690 2024-09-13T17:45:09.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73200a4ee0 con 0x7f732009d690 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f7310004950 con 0x7f732009d690 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73200a4430 con 0x7f732009d690 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 0x7f730c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f73100474e0 con 0x7f732009d690 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731ffff640 1 -- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 msgr2=0x7f730c0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:09.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.526+0000 7f731ffff640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 0x7f730c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:09.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.530+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f73200a4430 con 0x7f732009d690 2024-09-13T17:45:09.643 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.638+0000 7f30adca0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:45:09.694 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.686+0000 7f30adca0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:45:09.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.690+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7320138050 con 0x7f732009d690 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.690+0000 7f731d7fa640 1 -- 172.21.15.96:0/1024830833 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v57) v1 ==== 81+0+585 (secure 0 0 0) 0x7f7320138050 con 0x7f732009d690 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2016081916 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3931971068 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2331602584 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/3506364525 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.698 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/3506364525 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2147498054 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1429934372 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1435736606 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/4091393434 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/799438860 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:09.699 INFO:teuthology.orchestra.run.smithi096.stderr:listed 10 entries 2024-09-13T17:45:09.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 msgr2=0x7f730c0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:09.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 0x7f730c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 msgr2=0x7f73200a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:09.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f731000c390 tx=0x7f731000c850 comp rx=0 tx=0).stop 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 shutdown_connections 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f730c0354e0 0x7f730c0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 --2- 172.21.15.96:0/1024830833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f732009d690 0x7f73200a69e0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 >> 172.21.15.96:0/1024830833 conn(0x7f732006bc60 msgr2=0x7f7320139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 shutdown_connections 2024-09-13T17:45:09.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:09.694+0000 7f732700e640 1 -- 172.21.15.96:0/1024830833 wait complete. 2024-09-13T17:45:09.716 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:45:09.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:09.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:09.769 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[42248]: fuse finished with error 0 and tester_r 0 2024-09-13T17:45:09.770 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:45:09.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:09.771 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:45:09.820 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:09.820 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:09.877 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:09.921 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:45:09.921 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:45:09.922 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:45:09.930 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:09.922+0000 7f30adca0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:45:10.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.022+0000 7f57211f5640 1 Processor -- start 2024-09-13T17:45:10.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.022+0000 7f57211f5640 1 -- start start 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f571c136980 0x7f571c13b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c09ba60 con 0x7f571c09dc40 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571c09cf60 con 0x7f571c136980 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f571c136980 0x7f571c13b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f571c136980 0x7f571c13b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49190/0 (socket says 172.21.15.96:49190) 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571ad9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f571c09dc40 0x7f571c136270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57562/0 (socket says 172.21.15.96:57562) 2024-09-13T17:45:10.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 -- 172.21.15.96:0/1812828716 learned_addr learned my addr 172.21.15.96:0/1812828716 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:10.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 -- 172.21.15.96:0/1812828716 >> v1:172.21.15.96:6789/0 conn(0x7f571c09dc40 legacy=0x7f571c136270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 -- 172.21.15.96:0/1812828716 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c09c4e0 con 0x7f571c136980 2024-09-13T17:45:10.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f571a59c640 1 --2- 172.21.15.96:0/1812828716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c136980 0x7f571c13b0d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f57100097f0 tx=0x7f571001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7793e69f437adf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f5719d9b640 1 -- 172.21.15.96:0/1812828716 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57100ab530 con 0x7f571c136980 2024-09-13T17:45:10.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f5719d9b640 1 -- 172.21.15.96:0/1812828716 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57100a7050 con 0x7f571c136980 2024-09-13T17:45:10.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f5719d9b640 1 -- 172.21.15.96:0/1812828716 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57100ab7e0 con 0x7f571c136980 2024-09-13T17:45:10.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- 172.21.15.96:0/1812828716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c136980 msgr2=0x7f571c13b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 --2- 172.21.15.96:0/1812828716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c136980 0x7f571c13b0d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f57100097f0 tx=0x7f571001c450 comp rx=0 tx=0).stop 2024-09-13T17:45:10.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- 172.21.15.96:0/1812828716 shutdown_connections 2024-09-13T17:45:10.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 --2- 172.21.15.96:0/1812828716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c136980 0x7f571c13b0d0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- 172.21.15.96:0/1812828716 >> 172.21.15.96:0/1812828716 conn(0x7f571c06f350 msgr2=0x7f571c071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- 172.21.15.96:0/1812828716 shutdown_connections 2024-09-13T17:45:10.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.026+0000 7f57211f5640 1 -- 172.21.15.96:0/1812828716 wait complete. 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 Processor -- start 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 -- start start 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c06c4f0 con 0x7f571c09dc40 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49206/0 (socket says 172.21.15.96:49206) 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571ad9d640 1 -- 172.21.15.96:0/1563725590 learned_addr learned my addr 172.21.15.96:0/1563725590 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:10.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571ad9d640 1 -- 172.21.15.96:0/1563725590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100094d0 con 0x7f571c09dc40 2024-09-13T17:45:10.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571ad9d640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c3a0 tx=0x7f570c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f570c007410 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c007570 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f570c0047f0 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c0a3d90 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c0a51f0 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c136270 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f570c004950 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 0x7f57080379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571a59c640 1 -- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 msgr2=0x7f57080379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f570c047140 con 0x7f571c09dc40 2024-09-13T17:45:10.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.030+0000 7f571a59c640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 0x7f57080379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:10.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.034+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f570c0192c0 con 0x7f571c09dc40 2024-09-13T17:45:10.068 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.062+0000 7f30adca0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:10.178 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.170+0000 7f30adca0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:10.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.186+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f571c09ba60 con 0x7f571c09dc40 2024-09-13T17:45:10.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57037fe640 1 -- 172.21.15.96:0/1563725590 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+11118 (secure 0 0 0) 0x7f571c09ba60 con 0x7f571c09dc40 2024-09-13T17:45:10.196 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:10.197 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":57,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:06.004320+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:10.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 msgr2=0x7f57080379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:10.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 0x7f57080379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 msgr2=0x7f571c1cfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f570c00c3a0 tx=0x7f570c00c860 comp rx=0 tx=0).stop 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 shutdown_connections 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f57080354e0 0x7f57080379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 --2- 172.21.15.96:0/1563725590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f571c09dc40 0x7f571c1cfd90 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 >> 172.21.15.96:0/1563725590 conn(0x7f571c06f350 msgr2=0x7f571c135700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 shutdown_connections 2024-09-13T17:45:10.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.190+0000 7f57211f5640 1 -- 172.21.15.96:0/1563725590 wait complete. 2024-09-13T17:45:10.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:44:48.116392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.0299999713897705, 'score_stable': 1.0299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.0299999713897705, 'raw_score_stable': 1.0299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:44:49.284535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:10.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:10.241 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.234+0000 7f30adca0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:45:10.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.354+0000 7f8f640ec640 1 Processor -- start 2024-09-13T17:45:10.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.354+0000 7f8f640ec640 1 -- start start 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f5c132bd0 0x7f8f5c132fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5c0a3120 con 0x7f8f5c0a1370 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c071630 con 0x7f8f5c132bd0 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61e88640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f5c132bd0 0x7f8f5c132fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61687640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8f5c0a1370 0x7f8f5c0a17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57574/0 (socket says 172.21.15.96:57574) 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61e88640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f5c132bd0 0x7f8f5c132fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49220/0 (socket says 172.21.15.96:49220) 2024-09-13T17:45:10.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61687640 1 -- 172.21.15.96:0/789876945 learned_addr learned my addr 172.21.15.96:0/789876945 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:10.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f60e86640 1 -- 172.21.15.96:0/789876945 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153932246 0 0) 0x7f8f5c0a3120 con 0x7f8f5c0a1370 2024-09-13T17:45:10.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f60e86640 1 -- 172.21.15.96:0/789876945 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f48003530 con 0x7f8f5c0a1370 2024-09-13T17:45:10.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61e88640 1 -- 172.21.15.96:0/789876945 >> v1:172.21.15.96:6789/0 conn(0x7f8f5c0a1370 legacy=0x7f8f5c0a17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61e88640 1 -- 172.21.15.96:0/789876945 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c070970 con 0x7f8f5c132bd0 2024-09-13T17:45:10.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f61e88640 1 --2- 172.21.15.96:0/789876945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c132bd0 0x7f8f5c132fb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009d90 tx=0x7f8f5801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c7053fa3396808 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f60e86640 1 -- 172.21.15.96:0/789876945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f580a5650 con 0x7f8f5c132bd0 2024-09-13T17:45:10.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f60e86640 1 -- 172.21.15.96:0/789876945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580a57b0 con 0x7f8f5c132bd0 2024-09-13T17:45:10.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f60e86640 1 -- 172.21.15.96:0/789876945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f580a5a70 con 0x7f8f5c132bd0 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- 172.21.15.96:0/789876945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c132bd0 msgr2=0x7f8f5c132fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 --2- 172.21.15.96:0/789876945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c132bd0 0x7f8f5c132fb0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8f58009d90 tx=0x7f8f5801c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- 172.21.15.96:0/789876945 shutdown_connections 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 --2- 172.21.15.96:0/789876945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c132bd0 0x7f8f5c132fb0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- 172.21.15.96:0/789876945 >> 172.21.15.96:0/789876945 conn(0x7f8f5c0a6b40 msgr2=0x7f8f5c0a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.358+0000 7f8f640ec640 1 -- 172.21.15.96:0/789876945 shutdown_connections 2024-09-13T17:45:10.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- 172.21.15.96:0/789876945 wait complete. 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 Processor -- start 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- start start 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c09c7b0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61e88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61e88640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49234/0 (socket says 172.21.15.96:49234) 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61e88640 1 -- 172.21.15.96:0/546185789 learned_addr learned my addr 172.21.15.96:0/546185789 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61e88640 1 -- 172.21.15.96:0/546185789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f58009a70 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61e88640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8f580b0040 tx=0x7f8f58019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f580ad570 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f580ad6f0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c09b0e0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f580ad9d0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c09c050 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f8f580b4050 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c09bda0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 0x7f8f400379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61687640 1 -- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 msgr2=0x7f8f400379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:10.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f8f580e26b0 con 0x7f8f5c0a1370 2024-09-13T17:45:10.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.362+0000 7f8f61687640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 0x7f8f400379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:10.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.370+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f580c0950 con 0x7f8f5c0a1370 2024-09-13T17:45:10.378 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.370+0000 7f30adca0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:10.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8f5c09c580 con 0x7f8f5c0a1370 2024-09-13T17:45:10.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f527fc640 1 -- 172.21.15.96:0/546185789 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8f580c7020 con 0x7f8f5c0a1370 2024-09-13T17:45:10.528 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 msgr2=0x7f8f400379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 0x7f8f400379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 msgr2=0x7f8f5c09cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8f580b0040 tx=0x7f8f58019680 comp rx=0 tx=0).stop 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 shutdown_connections 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f8f400354e0 0x7f8f400379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 --2- 172.21.15.96:0/546185789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f5c0a1370 0x7f8f5c09cda0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 >> 172.21.15.96:0/546185789 conn(0x7f8f5c0a6b40 msgr2=0x7f8f5c131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 shutdown_connections 2024-09-13T17:45:10.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.522+0000 7f8f640ec640 1 -- 172.21.15.96:0/546185789 wait complete. 2024-09-13T17:45:10.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:10.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.658+0000 7fb29283d640 1 Processor -- start 2024-09-13T17:45:10.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.658+0000 7fb29283d640 1 -- start start 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.658+0000 7fb29283d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb28c099fe0 0x7fb28c099170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.658+0000 7fb29283d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c09a440 con 0x7fb28c09cb60 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.658+0000 7fb29283d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28c06a940 con 0x7fb28c099fe0 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb28c099fe0 0x7fb28c099170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb28c09cb60 0x7fb28c098a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57586/0 (socket says 172.21.15.96:57586) 2024-09-13T17:45:10.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb28c099fe0 0x7fb28c099170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49236/0 (socket says 172.21.15.96:49236) 2024-09-13T17:45:10.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28bfff640 1 -- 172.21.15.96:0/648844006 learned_addr learned my addr 172.21.15.96:0/648844006 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:10.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28b7fe640 1 -- 172.21.15.96:0/648844006 >> v1:172.21.15.96:6789/0 conn(0x7fb28c09cb60 legacy=0x7fb28c098a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28b7fe640 1 -- 172.21.15.96:0/648844006 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c09b920 con 0x7fb28c099fe0 2024-09-13T17:45:10.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28b7fe640 1 --2- 172.21.15.96:0/648844006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c099170 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb280009740 tx=0x7fb28001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd32f3f3054f78 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28affd640 1 -- 172.21.15.96:0/648844006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280004160 con 0x7fb28c099fe0 2024-09-13T17:45:10.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28affd640 1 -- 172.21.15.96:0/648844006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2800a7050 con 0x7fb28c099fe0 2024-09-13T17:45:10.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb28affd640 1 -- 172.21.15.96:0/648844006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb280004450 con 0x7fb28c099fe0 2024-09-13T17:45:10.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 -- 172.21.15.96:0/648844006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 msgr2=0x7fb28c099170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 --2- 172.21.15.96:0/648844006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c099170 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb280009740 tx=0x7fb28001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:10.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 -- 172.21.15.96:0/648844006 shutdown_connections 2024-09-13T17:45:10.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 --2- 172.21.15.96:0/648844006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c099170 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 -- 172.21.15.96:0/648844006 >> 172.21.15.96:0/648844006 conn(0x7fb28c06e020 msgr2=0x7fb28c070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 -- 172.21.15.96:0/648844006 shutdown_connections 2024-09-13T17:45:10.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.662+0000 7fb29283d640 1 -- 172.21.15.96:0/648844006 wait complete. 2024-09-13T17:45:10.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 Processor -- start 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 -- start start 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c1224c0 con 0x7fb28c099fe0 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49252/0 (socket says 172.21.15.96:49252) 2024-09-13T17:45:10.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28bfff640 1 -- 172.21.15.96:0/875095669 learned_addr learned my addr 172.21.15.96:0/875095669 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:10.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28bfff640 1 -- 172.21.15.96:0/875095669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280009420 con 0x7fb28c099fe0 2024-09-13T17:45:10.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28bfff640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fb28400cb20 tx=0x7fb28400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb28400abf0 con 0x7fb28c099fe0 2024-09-13T17:45:10.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28400ad50 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c071760 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb284012400 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c121be0 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fb284012560 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c098a60 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 0x7fb2780379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fb284047760 con 0x7fb28c099fe0 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28b7fe640 1 -- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 msgr2=0x7fb2780379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:10.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.666+0000 7fb28b7fe640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 0x7fb2780379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.670+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb28401ae20 con 0x7fb28c099fe0 2024-09-13T17:45:10.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.830+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb28c09a440 con 0x7fb28c099fe0 2024-09-13T17:45:10.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.830+0000 7fb288ff9640 1 -- 172.21.15.96:0/875095669 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb28c09a440 con 0x7fb28c099fe0 2024-09-13T17:45:10.836 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.830+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 msgr2=0x7fb2780379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.830+0000 7fb29283d640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 0x7fb2780379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 msgr2=0x7fb28c124180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fb28400cb20 tx=0x7fb28400a060 comp rx=0 tx=0).stop 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 shutdown_connections 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fb2780354e0 0x7fb2780379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 --2- 172.21.15.96:0/875095669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb28c099fe0 0x7fb28c124180 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 >> 172.21.15.96:0/875095669 conn(0x7fb28c06e020 msgr2=0x7fb28c0a61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 shutdown_connections 2024-09-13T17:45:10.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.834+0000 7fb29283d640 1 -- 172.21.15.96:0/875095669 wait complete. 2024-09-13T17:45:10.854 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:10.865 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.858+0000 7f30adca0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:45:10.940 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:10.934+0000 7f30adca0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:45:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c225b9640 1 Processor -- start 2024-09-13T17:45:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c225b9640 1 -- start start 2024-09-13T17:45:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c225b9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c1c09c930 0x7f5c1c098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c225b9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c1c09aee0 con 0x7f5c1c09cd10 2024-09-13T17:45:10.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c225b9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c1c09be70 con 0x7f5c1c09c930 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c1bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5c1c09c930 0x7f5c1c098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c1b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5c1c09cd10 0x7f5c1c098fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57594/0 (socket says 172.21.15.96:57594) 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.962+0000 7f5c1b7fe640 1 -- 172.21.15.96:0/1105995956 learned_addr learned my addr 172.21.15.96:0/1105995956 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1bfff640 1 -- 172.21.15.96:0/1105995956 >> v1:172.21.15.96:6789/0 conn(0x7f5c1c09cd10 legacy=0x7f5c1c098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1affd640 1 -- 172.21.15.96:0/1105995956 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918858670 0 0) 0x7f5c1c09aee0 con 0x7f5c1c09cd10 2024-09-13T17:45:10.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1bfff640 1 -- 172.21.15.96:0/1105995956 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c1c06ae90 con 0x7f5c1c09c930 2024-09-13T17:45:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1bfff640 1 --2- 172.21.15.96:0/1105995956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c098a60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009740 tx=0x7f5c0c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999a91a87f39bd4f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1affd640 1 -- 172.21.15.96:0/1105995956 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c0ab530 con 0x7f5c1c09c930 2024-09-13T17:45:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1affd640 1 -- 172.21.15.96:0/1105995956 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0a7050 con 0x7f5c1c09c930 2024-09-13T17:45:10.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.966+0000 7f5c1affd640 1 -- 172.21.15.96:0/1105995956 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c0ab7e0 con 0x7f5c1c09c930 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1105995956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 msgr2=0x7f5c1c098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1105995956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c098a60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009740 tx=0x7f5c0c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1105995956 shutdown_connections 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1105995956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c098a60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1105995956 >> 172.21.15.96:0/1105995956 conn(0x7f5c1c06e350 msgr2=0x7f5c1c070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1105995956 shutdown_connections 2024-09-13T17:45:10.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1105995956 wait complete. 2024-09-13T17:45:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 Processor -- start 2024-09-13T17:45:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- start start 2024-09-13T17:45:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1c124050 con 0x7f5c1c09c930 2024-09-13T17:45:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49278/0 (socket says 172.21.15.96:49278) 2024-09-13T17:45:10.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1bfff640 1 -- 172.21.15.96:0/1888359456 learned_addr learned my addr 172.21.15.96:0/1888359456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1bfff640 1 -- 172.21.15.96:0/1888359456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c009420 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1bfff640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c018fa0 tx=0x7f5c0c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c004160 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c0042c0 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1c1251f0 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0c003620 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c1c13bce0 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1c124190 con 0x7f5c1c09c930 2024-09-13T17:45:10.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5c0c003780 con 0x7f5c1c09c930 2024-09-13T17:45:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 0x7f5bfc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7f5c0c0e27f0 con 0x7f5c1c09c930 2024-09-13T17:45:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1b7fe640 1 -- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 msgr2=0x7f5bfc0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:10.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.970+0000 7f5c1b7fe640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 0x7f5bfc0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:10.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:10.978+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5c0c0b7450 con 0x7f5c1c09c930 2024-09-13T17:45:11.090 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.082+0000 7f30adca0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:45:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.134+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5c1c123000 con 0x7f5c1c09c930 2024-09-13T17:45:11.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c18ff9640 1 -- 172.21.15.96:0/1888359456 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5c0c0b0070 con 0x7f5c1c09c930 2024-09-13T17:45:11.143 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 msgr2=0x7f5bfc0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 0x7f5bfc0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 msgr2=0x7f5c1c1cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c018fa0 tx=0x7f5c0c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 shutdown_connections 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7f5bfc0354e0 0x7f5bfc0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 --2- 172.21.15.96:0/1888359456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c1c09c930 0x7f5c1c1cf680 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 >> 172.21.15.96:0/1888359456 conn(0x7f5c1c06e350 msgr2=0x7f5c1c0a2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 shutdown_connections 2024-09-13T17:45:11.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.138+0000 7f5c225b9640 1 -- 172.21.15.96:0/1888359456 wait complete. 2024-09-13T17:45:11.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:45:11.164 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.158+0000 7f30adca0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:45:11.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.278+0000 7fd1c5f16640 1 Processor -- start 2024-09-13T17:45:11.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.278+0000 7fd1c5f16640 1 -- start start 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1c5f16640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1c013af50 0x7fd1c013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1c5f16640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c0138050 con 0x7fd1c009d690 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1c5f16640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c009c410 con 0x7fd1c013af50 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1beffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1c013af50 0x7fd1c013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1beffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd1c013af50 0x7fd1c013d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49290/0 (socket says 172.21.15.96:49290) 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1beffd640 1 -- 172.21.15.96:0/2639881765 learned_addr learned my addr 172.21.15.96:0/2639881765 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1bf7fe640 1 --1- 172.21.15.96:0/2639881765 >> v1:172.21.15.96:6789/0 conn(0x7fd1c009d690 0x7fd1c0138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57596/0 (socket says 172.21.15.96:57596) 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1beffd640 1 -- 172.21.15.96:0/2639881765 >> v1:172.21.15.96:6789/0 conn(0x7fd1c009d690 legacy=0x7fd1c0138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1be7fc640 1 -- 172.21.15.96:0/2639881765 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163705271 0 0) 0x7fd1b0001120 con 0x7fd1c009d690 2024-09-13T17:45:11.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.282+0000 7fd1beffd640 1 -- 172.21.15.96:0/2639881765 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c009cba0 con 0x7fd1c013af50 2024-09-13T17:45:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1beffd640 1 --2- 172.21.15.96:0/2639881765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c013af50 0x7fd1c013d340 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80097f0 tx=0x7fd1a801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879cadbae032fe0e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1be7fc640 1 -- 172.21.15.96:0/2639881765 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a80a8900 con 0x7fd1c013af50 2024-09-13T17:45:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1be7fc640 1 -- 172.21.15.96:0/2639881765 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a80a8a60 con 0x7fd1c013af50 2024-09-13T17:45:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1be7fc640 1 -- 172.21.15.96:0/2639881765 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1a80a8d20 con 0x7fd1c013af50 2024-09-13T17:45:11.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/2639881765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c013af50 msgr2=0x7fd1c013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/2639881765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c013af50 0x7fd1c013d340 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80097f0 tx=0x7fd1a801ecd0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/2639881765 shutdown_connections 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/2639881765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c013af50 0x7fd1c013d340 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/2639881765 >> 172.21.15.96:0/2639881765 conn(0x7fd1c006bc60 msgr2=0x7fd1c006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/2639881765 shutdown_connections 2024-09-13T17:45:11.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/2639881765 wait complete. 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 Processor -- start 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- start start 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00a6f20 con 0x7fd1c009d690 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49304/0 (socket says 172.21.15.96:49304) 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1bf7fe640 1 -- 172.21.15.96:0/189853343 learned_addr learned my addr 172.21.15.96:0/189853343 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:11.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1bf7fe640 1 -- 172.21.15.96:0/189853343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80094d0 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1bf7fe640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c390 tx=0x7fd1b000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b0007410 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b0007590 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c00a4cb0 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1b00047f0 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c00a4ee0 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.286+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c00a4430 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.290+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 29) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fd1b0004950 con 0x7fd1c009d690 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.290+0000 7fd1b7fff640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 0x7fd1a00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:11.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.290+0000 7fd1beffd640 1 -- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 msgr2=0x7fd1a00379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/420999973 2024-09-13T17:45:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.290+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fd1b00474d0 con 0x7fd1c009d690 2024-09-13T17:45:11.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.290+0000 7fd1beffd640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 0x7fd1a00379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:11.299 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.294+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd1c00a4430 con 0x7fd1c009d690 2024-09-13T17:45:11.454 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.446+0000 7f30adca0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:45:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.466+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd1c0138050 con 0x7fd1c009d690 2024-09-13T17:45:11.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.466+0000 7fd1b7fff640 1 -- 172.21.15.96:0/189853343 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v57) v1 ==== 81+0+585 (secure 0 0 0) 0x7fd1c0138050 con 0x7fd1c009d690 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2016081916 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3931971068 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2331602584 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/3506364525 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/3506364525 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2147498054 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1429934372 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1435736606 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/4091393434 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.473 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/799438860 2024-09-14T17:45:06.004232+0000 2024-09-13T17:45:11.474 INFO:teuthology.orchestra.run.smithi096.stderr:listed 10 entries 2024-09-13T17:45:11.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 msgr2=0x7fd1a00379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:11.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 0x7fd1a00379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 msgr2=0x7fd1c00a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000c390 tx=0x7fd1b000c850 comp rx=0 tx=0).stop 2024-09-13T17:45:11.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 shutdown_connections 2024-09-13T17:45:11.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:6824/420999973,v1:172.21.15.96:6825/420999973] conn(0x7fd1a00354e0 0x7fd1a00379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 --2- 172.21.15.96:0/189853343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1c009d690 0x7fd1c00a69e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 >> 172.21.15.96:0/189853343 conn(0x7fd1c006bc60 msgr2=0x7fd1c0139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:11.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 shutdown_connections 2024-09-13T17:45:11.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.470+0000 7fd1c5f16640 1 -- 172.21.15.96:0/189853343 wait complete. 2024-09-13T17:45:11.491 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:45:11.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:11.491 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:11.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[42355]: fuse finished with error 0 and tester_r 0 2024-09-13T17:45:11.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:45:11.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:11.501 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:45:11.511 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.506+0000 7f30adca0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:45:11.601 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.594+0000 7f3036214640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.601 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.594+0000 7f3036214640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.601 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.594+0000 7f3036214640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.602 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.594+0000 7f3036214640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.602 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.594+0000 7f3036214640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.598+0000 7f303921a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.598+0000 7f303921a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.598+0000 7f303921a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.598+0000 7f303921a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.603 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:11.598+0000 7f303921a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:11.624 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.774+0000 7f8a19608640 1 Processor -- start 2024-09-13T17:45:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.774+0000 7f8a19608640 1 -- start start 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a1413af50 0x7f8a1413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a14138050 con 0x7f8a1409d690 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a1409c410 con 0x7f8a1413af50 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a0bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a1413af50 0x7f8a1413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a12ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8a1409d690 0x7f8a14138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57602/0 (socket says 172.21.15.96:57602) 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a12ffd640 1 -- 172.21.15.96:0/3915621014 learned_addr learned my addr 172.21.15.96:0/3915621014 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:11.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a0bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a1413af50 0x7f8a1413d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49424/0 (socket says 172.21.15.96:49424) 2024-09-13T17:45:11.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a0bfff640 1 -- 172.21.15.96:0/3915621014 >> v1:172.21.15.96:6789/0 conn(0x7f8a1409d690 legacy=0x7f8a14138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a0bfff640 1 -- 172.21.15.96:0/3915621014 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1409cba0 con 0x7f8a1413af50 2024-09-13T17:45:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a0bfff640 1 --2- 172.21.15.96:0/3915621014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1413af50 0x7f8a1413d340 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009740 tx=0x7f89fc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7063f785a4ebe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a127fc640 1 -- 172.21.15.96:0/3915621014 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89fc0a8840 con 0x7f8a1413af50 2024-09-13T17:45:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a127fc640 1 -- 172.21.15.96:0/3915621014 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc0a89a0 con 0x7f8a1413af50 2024-09-13T17:45:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a127fc640 1 -- 172.21.15.96:0/3915621014 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f89fc0a8c60 con 0x7f8a1413af50 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- 172.21.15.96:0/3915621014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1413af50 msgr2=0x7f8a1413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 --2- 172.21.15.96:0/3915621014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1413af50 0x7f8a1413d340 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009740 tx=0x7f89fc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- 172.21.15.96:0/3915621014 shutdown_connections 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 --2- 172.21.15.96:0/3915621014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1413af50 0x7f8a1413d340 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- 172.21.15.96:0/3915621014 >> 172.21.15.96:0/3915621014 conn(0x7f8a1406bc60 msgr2=0x7f8a1406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.778+0000 7f8a19608640 1 -- 172.21.15.96:0/3915621014 shutdown_connections 2024-09-13T17:45:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- 172.21.15.96:0/3915621014 wait complete. 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 Processor -- start 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- start start 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140a2d40 con 0x7f8a1409d690 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a12ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49428/0 (socket says 172.21.15.96:49428) 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a12ffd640 1 -- 172.21.15.96:0/3313637883 learned_addr learned my addr 172.21.15.96:0/3313637883 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a12ffd640 1 -- 172.21.15.96:0/3313637883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc009420 con 0x7f8a1409d690 2024-09-13T17:45:11.789 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a12ffd640 1 --2- 172.21.15.96:0/3313637883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c380 tx=0x7f8a0400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a04005c20 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a04005d80 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a141372e0 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a04007200 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140a5100 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f8a04007360 con 0x7f8a1409d690 2024-09-13T17:45:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140a4f10 con 0x7f8a1409d690 2024-09-13T17:45:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.782+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f8a04046810 con 0x7f8a1409d690 2024-09-13T17:45:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.790+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8a04007610 con 0x7f8a1409d690 2024-09-13T17:45:11.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.978+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a14138050 con 0x7f8a1409d690 2024-09-13T17:45:11.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a0b7fe640 1 -- 172.21.15.96:0/3313637883 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f8a14138050 con 0x7f8a1409d690 2024-09-13T17:45:11.987 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:11.988 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:45:11.988 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:45:11.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 msgr2=0x7f8a141c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:11.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 --2- 172.21.15.96:0/3313637883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c380 tx=0x7f8a0400c840 comp rx=0 tx=0).stop 2024-09-13T17:45:11.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 shutdown_connections 2024-09-13T17:45:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 --2- 172.21.15.96:0/3313637883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a1409d690 0x7f8a141c9550 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 >> 172.21.15.96:0/3313637883 conn(0x7f8a1406bc60 msgr2=0x7f8a14131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 shutdown_connections 2024-09-13T17:45:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:11.982+0000 7f8a19608640 1 -- 172.21.15.96:0/3313637883 wait complete. 2024-09-13T17:45:12.006 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f936e52f640 1 Processor -- start 2024-09-13T17:45:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f936e52f640 1 -- start start 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f936e52f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f93680aa580 0x7f936806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f936e52f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93680aa380 con 0x7f93680aa960 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f936e52f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936806d9d0 con 0x7f93680aa580 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f9367fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f93680aa580 0x7f936806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f93677fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f93680aa960 0x7f936813c0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:57606/0 (socket says 172.21.15.96:57606) 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f93677fe640 1 -- 172.21.15.96:0/1254910995 learned_addr learned my addr 172.21.15.96:0/1254910995 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f9367fff640 1 -- 172.21.15.96:0/1254910995 >> v1:172.21.15.96:6789/0 conn(0x7f93680aa960 legacy=0x7f936813c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f9367fff640 1 -- 172.21.15.96:0/1254910995 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936806ec10 con 0x7f93680aa580 2024-09-13T17:45:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.114+0000 7f9366ffd640 1 -- 172.21.15.96:0/1254910995 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2728203950 0 0) 0x7f93680aa380 con 0x7f93680aa960 2024-09-13T17:45:12.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 --2- 172.21.15.96:0/1254910995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f936806f650 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9358009740 tx=0x7f935801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c449837c5b8bcfac server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9366ffd640 1 -- 172.21.15.96:0/1254910995 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93580ab530 con 0x7f93680aa580 2024-09-13T17:45:12.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9366ffd640 1 -- 172.21.15.96:0/1254910995 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580a7050 con 0x7f93680aa580 2024-09-13T17:45:12.123 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9366ffd640 1 -- 172.21.15.96:0/1254910995 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93580ab7e0 con 0x7f93680aa580 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/1254910995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 msgr2=0x7f936806f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 --2- 172.21.15.96:0/1254910995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f936806f650 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f9358009740 tx=0x7f935801c390 comp rx=0 tx=0).stop 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/1254910995 shutdown_connections 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 --2- 172.21.15.96:0/1254910995 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f936806f650 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/1254910995 >> 172.21.15.96:0/1254910995 conn(0x7f936806bc60 msgr2=0x7f936806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/1254910995 shutdown_connections 2024-09-13T17:45:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/1254910995 wait complete. 2024-09-13T17:45:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 Processor -- start 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- start start 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368099c70 con 0x7f93680aa580 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49438/0 (socket says 172.21.15.96:49438) 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 -- 172.21.15.96:0/2988022043 learned_addr learned my addr 172.21.15.96:0/2988022043 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 -- 172.21.15.96:0/2988022043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009420 con 0x7f93680aa580 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9367fff640 1 --2- 172.21.15.96:0/2988022043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f9358019fc0 tx=0x7f935801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93580b7450 con 0x7f93680aa580 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580b75b0 con 0x7f93680aa580 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9368141530 con 0x7f93680aa580 2024-09-13T17:45:12.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f93580b7870 con 0x7f93680aa580 2024-09-13T17:45:12.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.118+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936809d860 con 0x7f93680aa580 2024-09-13T17:45:12.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.122+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936809a4e0 con 0x7f93680aa580 2024-09-13T17:45:12.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.122+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f93580b79d0 con 0x7f93680aa580 2024-09-13T17:45:12.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.122+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f93580e22b0 con 0x7f93680aa580 2024-09-13T17:45:12.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.126+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f93580bf770 con 0x7f93680aa580 2024-09-13T17:45:12.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.282+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936806f650 con 0x7f93680aa580 2024-09-13T17:45:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f9364ff9640 1 -- 172.21.15.96:0/2988022043 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+11555 (secure 0 0 0) 0x7f93580b0020 con 0x7f93680aa580 2024-09-13T17:45:12.291 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:12.292 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":58,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:11.531861+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/692121072":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2400367826":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2022112443":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6825/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6824/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1206016791":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3403848071":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 msgr2=0x7f9368098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 --2- 172.21.15.96:0/2988022043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f9358019fc0 tx=0x7f935801abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 shutdown_connections 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 --2- 172.21.15.96:0/2988022043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f93680aa580 0x7f9368098a60 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 >> 172.21.15.96:0/2988022043 conn(0x7f936806bc60 msgr2=0x7f93681394b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 shutdown_connections 2024-09-13T17:45:12.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.286+0000 7f936e52f640 1 -- 172.21.15.96:0/2988022043 wait complete. 2024-09-13T17:45:12.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:44:48.116392+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.0299999713897705, 'score_stable': 1.0299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.0299999713897705, 'raw_score_stable': 1.0299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:44:49.284535+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:12.310 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.426+0000 7f83564c7640 1 Processor -- start 2024-09-13T17:45:12.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.426+0000 7f83564c7640 1 -- start start 2024-09-13T17:45:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.426+0000 7f83564c7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8350134e90 0x7f8350071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.426+0000 7f83564c7640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8350070de0 con 0x7f8350135270 2024-09-13T17:45:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.426+0000 7f83564c7640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8350071d60 con 0x7f8350134e90 2024-09-13T17:45:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8350134e90 0x7f8350071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8350134e90 0x7f8350071530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60392/0 (socket says 172.21.15.96:60392) 2024-09-13T17:45:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8350135270 0x7f835006dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60934/0 (socket says 172.21.15.96:60934) 2024-09-13T17:45:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 -- 172.21.15.96:0/1379235326 learned_addr learned my addr 172.21.15.96:0/1379235326 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 -- 172.21.15.96:0/1379235326 >> v1:172.21.15.96:6789/0 conn(0x7f8350135270 legacy=0x7f835006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 -- 172.21.15.96:0/1379235326 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835006f110 con 0x7f8350134e90 2024-09-13T17:45:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834ffff640 1 --2- 172.21.15.96:0/1379235326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f8350071530 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f83380097f0 tx=0x7f833801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c75b6a7367f443c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834effd640 1 -- 172.21.15.96:0/1379235326 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83380ab530 con 0x7f8350134e90 2024-09-13T17:45:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834effd640 1 -- 172.21.15.96:0/1379235326 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83380a7050 con 0x7f8350134e90 2024-09-13T17:45:12.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f834effd640 1 -- 172.21.15.96:0/1379235326 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83380ab7e0 con 0x7f8350134e90 2024-09-13T17:45:12.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 -- 172.21.15.96:0/1379235326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 msgr2=0x7f8350071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 --2- 172.21.15.96:0/1379235326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f8350071530 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f83380097f0 tx=0x7f833801c450 comp rx=0 tx=0).stop 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 -- 172.21.15.96:0/1379235326 shutdown_connections 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 --2- 172.21.15.96:0/1379235326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f8350071530 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 -- 172.21.15.96:0/1379235326 >> 172.21.15.96:0/1379235326 conn(0x7f8350098a60 msgr2=0x7f835009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 -- 172.21.15.96:0/1379235326 shutdown_connections 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.430+0000 7f83564c7640 1 -- 172.21.15.96:0/1379235326 wait complete. 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 Processor -- start 2024-09-13T17:45:12.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 -- start start 2024-09-13T17:45:12.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835013cb00 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60404/0 (socket says 172.21.15.96:60404) 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834ffff640 1 -- 172.21.15.96:0/802710161 learned_addr learned my addr 172.21.15.96:0/802710161 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834ffff640 1 -- 172.21.15.96:0/802710161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83380094d0 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834ffff640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f833801a080 tx=0x7f833801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83380b6420 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83380b6580 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83380b6840 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83500a1700 con 0x7f8350134e90 2024-09-13T17:45:12.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83500a6590 con 0x7f8350134e90 2024-09-13T17:45:12.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 30) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f83380b2040 con 0x7f8350134e90 2024-09-13T17:45:12.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8350071530 con 0x7f8350134e90 2024-09-13T17:45:12.442 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.434+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f83380e1990 con 0x7f8350134e90 2024-09-13T17:45:12.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.438+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f83380b69a0 con 0x7f8350134e90 2024-09-13T17:45:12.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.534+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f83380b6cd0 con 0x7f8350134e90 2024-09-13T17:45:12.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.534+0000 7f834cff9640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 0x7f833003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.534+0000 7f834f7fe640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 0x7f833003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.538+0000 7f834f7fe640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 0x7f833003bfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83400015d0 tx=0x7f8340007a30 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.596 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.590+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83500a2310 con 0x7f8350134e90 2024-09-13T17:45:12.597 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.590+0000 7f834cff9640 1 -- 172.21.15.96:0/802710161 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83380ba030 con 0x7f8350134e90 2024-09-13T17:45:12.597 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 msgr2=0x7f833003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 0x7f833003bfa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f83400015d0 tx=0x7f8340007a30 comp rx=0 tx=0).stop 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 msgr2=0x7f83501d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f833801a080 tx=0x7f833801ac90 comp rx=0 tx=0).stop 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 shutdown_connections 2024-09-13T17:45:12.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8330039bb0 0x7f833003bfa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 --2- 172.21.15.96:0/802710161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8350134e90 0x7f83501d0360 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 >> 172.21.15.96:0/802710161 conn(0x7f8350098a60 msgr2=0x7f8350133f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 shutdown_connections 2024-09-13T17:45:12.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.594+0000 7f83564c7640 1 -- 172.21.15.96:0/802710161 wait complete. 2024-09-13T17:45:12.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:12.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c2032a640 1 Processor -- start 2024-09-13T17:45:12.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c2032a640 1 -- start start 2024-09-13T17:45:12.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c2032a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9c1809e820 0x7f9c1813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c2032a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c180a68b0 con 0x7f9c18134e70 2024-09-13T17:45:12.734 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c2032a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c1809ddc0 con 0x7f9c1809e820 2024-09-13T17:45:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c1e0c6640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9c18134e70 0x7f9c18135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60944/0 (socket says 172.21.15.96:60944) 2024-09-13T17:45:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c1d8c5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9c1809e820 0x7f9c1813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c1e0c6640 1 -- 172.21.15.96:0/898041637 learned_addr learned my addr 172.21.15.96:0/898041637 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c1d8c5640 1 -- 172.21.15.96:0/898041637 >> v1:172.21.15.96:6789/0 conn(0x7f9c18134e70 legacy=0x7f9c18135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.735 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.726+0000 7f9c1d8c5640 1 -- 172.21.15.96:0/898041637 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c18136970 con 0x7f9c1809e820 2024-09-13T17:45:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c1d8c5640 1 --2- 172.21.15.96:0/898041637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c1813ae50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009740 tx=0x7f9c0801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d63f74f26aa2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c1d0c4640 1 -- 172.21.15.96:0/898041637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c080ab530 con 0x7f9c1809e820 2024-09-13T17:45:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c1d0c4640 1 -- 172.21.15.96:0/898041637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c080a7050 con 0x7f9c1809e820 2024-09-13T17:45:12.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c1d0c4640 1 -- 172.21.15.96:0/898041637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c080ab7e0 con 0x7f9c1809e820 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- 172.21.15.96:0/898041637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 msgr2=0x7f9c1813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 --2- 172.21.15.96:0/898041637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c1813ae50 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9c08009740 tx=0x7f9c0801c420 comp rx=0 tx=0).stop 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- 172.21.15.96:0/898041637 shutdown_connections 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 --2- 172.21.15.96:0/898041637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c1813ae50 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- 172.21.15.96:0/898041637 >> 172.21.15.96:0/898041637 conn(0x7f9c18098a60 msgr2=0x7f9c1809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.738 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- 172.21.15.96:0/898041637 shutdown_connections 2024-09-13T17:45:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- 172.21.15.96:0/898041637 wait complete. 2024-09-13T17:45:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 Processor -- start 2024-09-13T17:45:12.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.730+0000 7f9c2032a640 1 -- start start 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c2032a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c2032a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180a3ee0 con 0x7f9c1809e820 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1e0c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1e0c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60420/0 (socket says 172.21.15.96:60420) 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1e0c6640 1 -- 172.21.15.96:0/3681505846 learned_addr learned my addr 172.21.15.96:0/3681505846 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1e0c6640 1 -- 172.21.15.96:0/3681505846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c08009420 con 0x7f9c1809e820 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1e0c6640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000ca90 tx=0x7f9c1000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.740 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c10004ca0 con 0x7f9c1809e820 2024-09-13T17:45:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10004e00 con 0x7f9c1809e820 2024-09-13T17:45:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c180a1390 con 0x7f9c1809e820 2024-09-13T17:45:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9c1000aba0 con 0x7f9c1809e820 2024-09-13T17:45:12.741 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c180a2030 con 0x7f9c1809e820 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c180008e0 con 0x7f9c1809e820 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f9c1000ad00 con 0x7f9c1809e820 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 0x7f9bfc037710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f9c10047630 con 0x7f9c1809e820 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1d8c5640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 0x7f9bfc037710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:12.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.734+0000 7f9c1d8c5640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 0x7f9bfc037710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ad000 tx=0x7f9c08008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:12.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.738+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9c10010300 con 0x7f9c1809e820 2024-09-13T17:45:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c180a61d0 con 0x7f9c1809e820 2024-09-13T17:45:12.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c0effd640 1 -- 172.21.15.96:0/3681505846 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9c10018070 con 0x7f9c1809e820 2024-09-13T17:45:12.904 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 msgr2=0x7f9bfc037710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 0x7f9bfc037710 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ad000 tx=0x7f9c08008040 comp rx=0 tx=0).stop 2024-09-13T17:45:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 msgr2=0x7f9c181cda00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000ca90 tx=0x7f9c1000cf50 comp rx=0 tx=0).stop 2024-09-13T17:45:12.906 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 shutdown_connections 2024-09-13T17:45:12.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9bfc035250 0x7f9bfc037710 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9c080ad000 tx=0x7f9c08008040 comp rx=0 tx=0).stop 2024-09-13T17:45:12.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 --2- 172.21.15.96:0/3681505846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c1809e820 0x7f9c181cda00 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:12.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 >> 172.21.15.96:0/3681505846 conn(0x7f9c18098a60 msgr2=0x7f9c18134070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:12.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 shutdown_connections 2024-09-13T17:45:12.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:12.898+0000 7f9c2032a640 1 -- 172.21.15.96:0/3681505846 wait complete. 2024-09-13T17:45:12.923 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febccf7d640 1 Processor -- start 2024-09-13T17:45:13.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febccf7d640 1 -- start start 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febccf7d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7febc80a1e60 0x7febc813d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febccf7d640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc809eca0 con 0x7febc80a1370 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febccf7d640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc809d440 con 0x7febc80a1e60 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febc659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7febc80a1370 0x7febc80a1750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60954/0 (socket says 172.21.15.96:60954) 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febc5d9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7febc80a1e60 0x7febc813d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febc659c640 1 -- 172.21.15.96:0/2711007952 learned_addr learned my addr 172.21.15.96:0/2711007952 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febc5d9b640 1 -- 172.21.15.96:0/2711007952 >> v1:172.21.15.96:6789/0 conn(0x7febc80a1370 legacy=0x7febc80a1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.042+0000 7febc5d9b640 1 -- 172.21.15.96:0/2711007952 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80a28b0 con 0x7febc80a1e60 2024-09-13T17:45:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc5d9b640 1 --2- 172.21.15.96:0/2711007952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1e60 0x7febc813d490 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7febbc009740 tx=0x7febbc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7357a25e2f6cc7ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc559a640 1 -- 172.21.15.96:0/2711007952 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febbc0ab530 con 0x7febc80a1e60 2024-09-13T17:45:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc559a640 1 -- 172.21.15.96:0/2711007952 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc0a7050 con 0x7febc80a1e60 2024-09-13T17:45:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc559a640 1 -- 172.21.15.96:0/2711007952 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febbc0ab7e0 con 0x7febc80a1e60 2024-09-13T17:45:13.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/2711007952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1e60 msgr2=0x7febc813d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 --2- 172.21.15.96:0/2711007952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1e60 0x7febc813d490 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7febbc009740 tx=0x7febbc01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/2711007952 shutdown_connections 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 --2- 172.21.15.96:0/2711007952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1e60 0x7febc813d490 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/2711007952 >> 172.21.15.96:0/2711007952 conn(0x7febc80a6b40 msgr2=0x7febc80a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/2711007952 shutdown_connections 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/2711007952 wait complete. 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 Processor -- start 2024-09-13T17:45:13.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- start start 2024-09-13T17:45:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc811da80 con 0x7febc80a1370 2024-09-13T17:45:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60436/0 (socket says 172.21.15.96:60436) 2024-09-13T17:45:13.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc659c640 1 -- 172.21.15.96:0/3244043015 learned_addr learned my addr 172.21.15.96:0/3244043015 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc659c640 1 -- 172.21.15.96:0/3244043015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009420 con 0x7febc80a1370 2024-09-13T17:45:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febc659c640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7febb800c3a0 tx=0x7febb800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb8005c20 con 0x7febc80a1370 2024-09-13T17:45:13.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb8005d80 con 0x7febc80a1370 2024-09-13T17:45:13.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc8071a70 con 0x7febc80a1370 2024-09-13T17:45:13.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7febb8007200 con 0x7febc80a1370 2024-09-13T17:45:13.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.046+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc811e990 con 0x7febc80a1370 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7febb8007360 con 0x7febc80a1370 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc80a1750 con 0x7febc80a1370 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febb6ffd640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 0x7feba8037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7febb80469a0 con 0x7febc80a1370 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febc5d9b640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 0x7feba8037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.050+0000 7febc5d9b640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 0x7feba8037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febbc0ad000 tx=0x7febbc008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.054+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7febb8007610 con 0x7febc80a1370 2024-09-13T17:45:13.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.210+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7febc809eca0 con 0x7febc80a1370 2024-09-13T17:45:13.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febb6ffd640 1 -- 172.21.15.96:0/3244043015 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7febc809eca0 con 0x7febc80a1370 2024-09-13T17:45:13.219 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:13.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 msgr2=0x7feba8037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 0x7feba8037760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7febbc0ad000 tx=0x7febbc008040 comp rx=0 tx=0).stop 2024-09-13T17:45:13.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 msgr2=0x7febc811e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7febb800c3a0 tx=0x7febb800c860 comp rx=0 tx=0).stop 2024-09-13T17:45:13.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 shutdown_connections 2024-09-13T17:45:13.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feba80352a0 0x7feba8037760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 --2- 172.21.15.96:0/3244043015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febc80a1370 0x7febc811e450 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 >> 172.21.15.96:0/3244043015 conn(0x7febc80a6b40 msgr2=0x7febc80a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 shutdown_connections 2024-09-13T17:45:13.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.214+0000 7febccf7d640 1 -- 172.21.15.96:0/3244043015 wait complete. 2024-09-13T17:45:13.238 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:13.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.346+0000 7f19b2e08640 1 Processor -- start 2024-09-13T17:45:13.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.346+0000 7f19b2e08640 1 -- start start 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b2e08640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19ac134e90 0x7f19ac071530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b2e08640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ac070de0 con 0x7f19ac135270 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b2e08640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ac071d60 con 0x7f19ac134e90 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19abfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f19ac135270 0x7f19ac06dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60970/0 (socket says 172.21.15.96:60970) 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b0ba4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f19ac134e90 0x7f19ac071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19abfff640 1 -- 172.21.15.96:0/1033792498 learned_addr learned my addr 172.21.15.96:0/1033792498 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:13.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b0ba4640 1 -- 172.21.15.96:0/1033792498 >> v1:172.21.15.96:6789/0 conn(0x7f19ac135270 legacy=0x7f19ac06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.356 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b0ba4640 1 -- 172.21.15.96:0/1033792498 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac06f110 con 0x7f19ac134e90 2024-09-13T17:45:13.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19b0ba4640 1 --2- 172.21.15.96:0/1033792498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac071530 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009740 tx=0x7f19a001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9e58c2d934f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19ab7fe640 1 -- 172.21.15.96:0/1033792498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00ab530 con 0x7f19ac134e90 2024-09-13T17:45:13.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19ab7fe640 1 -- 172.21.15.96:0/1033792498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a00a7050 con 0x7f19ac134e90 2024-09-13T17:45:13.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.350+0000 7f19ab7fe640 1 -- 172.21.15.96:0/1033792498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00ab7e0 con 0x7f19ac134e90 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/1033792498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 msgr2=0x7f19ac071530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 --2- 172.21.15.96:0/1033792498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac071530 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009740 tx=0x7f19a001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/1033792498 shutdown_connections 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 --2- 172.21.15.96:0/1033792498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac071530 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/1033792498 >> 172.21.15.96:0/1033792498 conn(0x7f19ac098a60 msgr2=0x7f19ac09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/1033792498 shutdown_connections 2024-09-13T17:45:13.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/1033792498 wait complete. 2024-09-13T17:45:13.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 Processor -- start 2024-09-13T17:45:13.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- start start 2024-09-13T17:45:13.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ac0a41e0 con 0x7f19ac134e90 2024-09-13T17:45:13.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b0ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b0ba4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60452/0 (socket says 172.21.15.96:60452) 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b0ba4640 1 -- 172.21.15.96:0/2196634406 learned_addr learned my addr 172.21.15.96:0/2196634406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b0ba4640 1 -- 172.21.15.96:0/2196634406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a0009420 con 0x7f19ac134e90 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b0ba4640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f19a0019fc0 tx=0x7f19a001abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00b7450 con 0x7f19ac134e90 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a00b75b0 con 0x7f19ac134e90 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ac0a52a0 con 0x7f19ac134e90 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19a00b7870 con 0x7f19ac134e90 2024-09-13T17:45:13.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ac13d000 con 0x7f19ac134e90 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ac0a5530 con 0x7f19ac134e90 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.354+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f19a00b79d0 con 0x7f19ac134e90 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.358+0000 7f19a97fa640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 0x7f1990037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.358+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f19a00e1f90 con 0x7f19ac134e90 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.358+0000 7f19abfff640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 0x7f1990037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.358+0000 7f19abfff640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 0x7f1990037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1998005890 tx=0x7f1998007a90 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.362+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f19ac0a5530 con 0x7f19ac134e90 2024-09-13T17:45:13.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.538+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19ac0a5530 con 0x7f19ac134e90 2024-09-13T17:45:13.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.538+0000 7f19a97fa640 1 -- 172.21.15.96:0/2196634406 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f19ac0a5530 con 0x7f19ac134e90 2024-09-13T17:45:13.544 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:13.544 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:45:13.545 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 msgr2=0x7f1990037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 0x7f1990037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1998005890 tx=0x7f1998007a90 comp rx=0 tx=0).stop 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 msgr2=0x7f19ac0a1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f19a0019fc0 tx=0x7f19a001abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 shutdown_connections 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f19900352a0 0x7f1990037760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 --2- 172.21.15.96:0/2196634406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f19ac134e90 0x7f19ac0a1b40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 >> 172.21.15.96:0/2196634406 conn(0x7f19ac098a60 msgr2=0x7f19ac133eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 shutdown_connections 2024-09-13T17:45:13.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.542+0000 7f19b2e08640 1 -- 172.21.15.96:0/2196634406 wait complete. 2024-09-13T17:45:13.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:13.581 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:13.574+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbb21b5640 1 Processor -- start 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbb21b5640 1 -- start start 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbb21b5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcbac132940 0x7fcbac132d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbb21b5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac1326f0 con 0x7fcbac133260 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbb21b5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac132500 con 0x7fcbac132940 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbab7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fcbac132940 0x7fcbac132d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcba3fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fcbac133260 0x7fcbac069f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60982/0 (socket says 172.21.15.96:60982) 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcba3fff640 1 -- 172.21.15.96:0/1745144006 learned_addr learned my addr 172.21.15.96:0/1745144006 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:13.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbab7fe640 1 -- 172.21.15.96:0/1745144006 >> v1:172.21.15.96:6789/0 conn(0x7fcbac133260 legacy=0x7fcbac069f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbab7fe640 1 -- 172.21.15.96:0/1745144006 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac09cfd0 con 0x7fcbac132940 2024-09-13T17:45:13.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.666+0000 7fcbaaffd640 1 -- 172.21.15.96:0/1745144006 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 722116511 0 0) 0x7fcbac1326f0 con 0x7fcbac133260 2024-09-13T17:45:13.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbab7fe640 1 --2- 172.21.15.96:0/1745144006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac132d20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009740 tx=0x7fcb9c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941833b5b1777810 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbaaffd640 1 -- 172.21.15.96:0/1745144006 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb9c0ab400 con 0x7fcbac132940 2024-09-13T17:45:13.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbaaffd640 1 -- 172.21.15.96:0/1745144006 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c0a7050 con 0x7fcbac132940 2024-09-13T17:45:13.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbaaffd640 1 -- 172.21.15.96:0/1745144006 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb9c0ab6b0 con 0x7fcbac132940 2024-09-13T17:45:13.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- 172.21.15.96:0/1745144006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 msgr2=0x7fcbac132d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/1745144006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac132d20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009740 tx=0x7fcb9c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- 172.21.15.96:0/1745144006 shutdown_connections 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/1745144006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac132d20 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- 172.21.15.96:0/1745144006 >> 172.21.15.96:0/1745144006 conn(0x7fcbac06e020 msgr2=0x7fcbac070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- 172.21.15.96:0/1745144006 shutdown_connections 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- 172.21.15.96:0/1745144006 wait complete. 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 Processor -- start 2024-09-13T17:45:13.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- start start 2024-09-13T17:45:13.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbb21b5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0712c0 con 0x7fcbac132940 2024-09-13T17:45:13.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60466/0 (socket says 172.21.15.96:60466) 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbab7fe640 1 -- 172.21.15.96:0/803819845 learned_addr learned my addr 172.21.15.96:0/803819845 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.670+0000 7fcbab7fe640 1 -- 172.21.15.96:0/803819845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009420 con 0x7fcbac132940 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcbab7fe640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c018fa0 tx=0x7fcb9c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb9c0abd70 con 0x7fcbac132940 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c004020 con 0x7fcbac132940 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac099f60 con 0x7fcbac132940 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcb9c004190 con 0x7fcbac132940 2024-09-13T17:45:13.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac09a780 con 0x7fcbac132940 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fcb9c0b2050 con 0x7fcbac132940 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac09a9b0 con 0x7fcbac132940 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 0x7fcb8c037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7fcb9c0e1970 con 0x7fcbac132940 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba3fff640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 0x7fcb8c037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:13.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.674+0000 7fcba3fff640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 0x7fcb8c037760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb94007fd0 tx=0x7fcb94007920 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:13.685 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.678+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fcbac09a9b0 con 0x7fcbac132940 2024-09-13T17:45:13.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.838+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbac09a9b0 con 0x7fcbac132940 2024-09-13T17:45:13.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.838+0000 7fcba8ff9640 1 -- 172.21.15.96:0/803819845 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+11555 (secure 0 0 0) 0x7fcbac09a9b0 con 0x7fcbac132940 2024-09-13T17:45:13.847 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:13.847 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":58,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:11.531861+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/692121072":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2400367826":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2022112443":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6825/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6824/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1206016791":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3403848071":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 msgr2=0x7fcb8c037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 0x7fcb8c037760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcb94007fd0 tx=0x7fcb94007920 comp rx=0 tx=0).stop 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 msgr2=0x7fcbac13ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c018fa0 tx=0x7fcb9c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 shutdown_connections 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fcb8c0352a0 0x7fcb8c037760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 --2- 172.21.15.96:0/803819845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcbac132940 0x7fcbac13ac90 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 >> 172.21.15.96:0/803819845 conn(0x7fcbac06e020 msgr2=0x7fcbac12f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 shutdown_connections 2024-09-13T17:45:13.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:13.842+0000 7fcbb21b5640 1 -- 172.21.15.96:0/803819845 wait complete. 2024-09-13T17:45:13.865 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:45:13.865 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28c184c640 1 Processor -- start 2024-09-13T17:45:14.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28c184c640 1 -- start start 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28c184c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f28bc09dc40 0x7f28bc136270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28c184c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28bc09ba60 con 0x7f28bc1367b0 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28c184c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28bc09cf60 con 0x7f28bc09dc40 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28baffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f28bc09dc40 0x7f28bc136270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28ba7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f28bc1367b0 0x7f28bc099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:60986/0 (socket says 172.21.15.96:60986) 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28ba7fc640 1 -- 172.21.15.96:0/2743997706 learned_addr learned my addr 172.21.15.96:0/2743997706 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:14.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28baffd640 1 -- 172.21.15.96:0/2743997706 >> v1:172.21.15.96:6789/0 conn(0x7f28bc1367b0 legacy=0x7f28bc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28baffd640 1 -- 172.21.15.96:0/2743997706 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28bc09c4e0 con 0x7f28bc09dc40 2024-09-13T17:45:14.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.022+0000 7f28b9ffb640 1 -- 172.21.15.96:0/2743997706 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3102791941 0 0) 0x7f28b0001120 con 0x7f28bc1367b0 2024-09-13T17:45:14.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 --2- 172.21.15.96:0/2743997706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc136270 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009740 tx=0x7f28ac01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2ad28c3a2294dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28b9ffb640 1 -- 172.21.15.96:0/2743997706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28ac0ab530 con 0x7f28bc09dc40 2024-09-13T17:45:14.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28b9ffb640 1 -- 172.21.15.96:0/2743997706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac0a7050 con 0x7f28bc09dc40 2024-09-13T17:45:14.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28b9ffb640 1 -- 172.21.15.96:0/2743997706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28ac0ab7e0 con 0x7f28bc09dc40 2024-09-13T17:45:14.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/2743997706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 msgr2=0x7f28bc136270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 --2- 172.21.15.96:0/2743997706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc136270 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009740 tx=0x7f28ac01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:14.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/2743997706 shutdown_connections 2024-09-13T17:45:14.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 --2- 172.21.15.96:0/2743997706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc136270 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/2743997706 >> 172.21.15.96:0/2743997706 conn(0x7f28bc06f350 msgr2=0x7f28bc071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/2743997706 shutdown_connections 2024-09-13T17:45:14.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/2743997706 wait complete. 2024-09-13T17:45:14.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 Processor -- start 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- start start 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28bc0a5190 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60474/0 (socket says 172.21.15.96:60474) 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 -- 172.21.15.96:0/924513414 learned_addr learned my addr 172.21.15.96:0/924513414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 -- 172.21.15.96:0/924513414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac009420 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28baffd640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f28ac019fc0 tx=0x7f28ac01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28ac004150 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ac0042b0 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28bc0a2f80 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f28ac003620 con 0x7f28bc09dc40 2024-09-13T17:45:14.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.026+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28bc0a4200 con 0x7f28bc09dc40 2024-09-13T17:45:14.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f28ac0b2050 con 0x7f28bc09dc40 2024-09-13T17:45:14.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28bc0a2630 con 0x7f28bc09dc40 2024-09-13T17:45:14.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28a3fff640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 0x7f28a8037710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f28ac0e1d40 con 0x7f28bc09dc40 2024-09-13T17:45:14.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28ba7fc640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 0x7f28a8037710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.030+0000 7f28ba7fc640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 0x7f28a8037710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c10 tx=0x7f28b0001320 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.034+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f28ac0e5030 con 0x7f28bc09dc40 2024-09-13T17:45:14.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.234+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28bc137120 con 0x7f28bc09dc40 2024-09-13T17:45:14.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.234+0000 7f28a3fff640 1 -- 172.21.15.96:0/924513414 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f28ac0b0070 con 0x7f28bc09dc40 2024-09-13T17:45:14.242 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:14.242 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:45:14.243 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 msgr2=0x7f28a8037710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 0x7f28a8037710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c10 tx=0x7f28b0001320 comp rx=0 tx=0).stop 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 msgr2=0x7f28bc0a31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f28ac019fc0 tx=0x7f28ac01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 shutdown_connections 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f28a8035250 0x7f28a8037710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 --2- 172.21.15.96:0/924513414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28bc09dc40 0x7f28bc0a31e0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 >> 172.21.15.96:0/924513414 conn(0x7f28bc06f350 msgr2=0x7f28bc071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 shutdown_connections 2024-09-13T17:45:14.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.238+0000 7f28c184c640 1 -- 172.21.15.96:0/924513414 wait complete. 2024-09-13T17:45:14.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:14.264 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:14.258+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:14.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2ba5db0640 1 Processor -- start 2024-09-13T17:45:14.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2ba5db0640 1 -- start start 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2ba5db0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ba009d690 0x7f2ba0138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2ba5db0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ba0138050 con 0x7f2ba01387d0 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2ba5db0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ba009c410 con 0x7f2ba009d690 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2b9f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ba009d690 0x7f2ba0138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2b9f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ba009d690 0x7f2ba0138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60484/0 (socket says 172.21.15.96:60484) 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2b9f7fe640 1 -- 172.21.15.96:0/3818101920 learned_addr learned my addr 172.21.15.96:0/3818101920 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:14.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2b9f7fe640 1 -- 172.21.15.96:0/3818101920 >> v1:172.21.15.96:6789/0 conn(0x7f2ba01387d0 legacy=0x7f2ba0134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.374+0000 7f2b9f7fe640 1 -- 172.21.15.96:0/3818101920 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba009cba0 con 0x7f2ba009d690 2024-09-13T17:45:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 --2- 172.21.15.96:0/3818101920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba0138290 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2b900097f0 tx=0x7f2b9001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27af43fd2874ca23 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9e7fc640 1 -- 172.21.15.96:0/3818101920 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b900a8900 con 0x7f2ba009d690 2024-09-13T17:45:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9e7fc640 1 -- 172.21.15.96:0/3818101920 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b900a8a60 con 0x7f2ba009d690 2024-09-13T17:45:14.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9e7fc640 1 -- 172.21.15.96:0/3818101920 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b900a8d20 con 0x7f2ba009d690 2024-09-13T17:45:14.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/3818101920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 msgr2=0x7f2ba0138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/3818101920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba0138290 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f2b900097f0 tx=0x7f2b9001ecd0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/3818101920 shutdown_connections 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/3818101920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba0138290 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/3818101920 >> 172.21.15.96:0/3818101920 conn(0x7f2ba006bc60 msgr2=0x7f2ba006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/3818101920 shutdown_connections 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/3818101920 wait complete. 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 Processor -- start 2024-09-13T17:45:14.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- start start 2024-09-13T17:45:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba00a5d10 con 0x7f2ba009d690 2024-09-13T17:45:14.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60488/0 (socket says 172.21.15.96:60488) 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 -- 172.21.15.96:0/886167053 learned_addr learned my addr 172.21.15.96:0/886167053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 -- 172.21.15.96:0/886167053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b900094d0 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b9f7fe640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2b9001c900 tx=0x7f2b9001ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b900bf070 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b900b09b0 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba00a4470 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2b900b0c70 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.378+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba009f490 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 31) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f2b900b7070 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba009bc00 con 0x7f2ba009d690 2024-09-13T17:45:14.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2b87fff640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 0x7f2b8c037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f2b900e4f90 con 0x7f2ba009d690 2024-09-13T17:45:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2b9effd640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 0x7f2b8c037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.382+0000 7f2b9effd640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 0x7f2b8c037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0067370 tx=0x7f2b940079e0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.386+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2ba009bc00 con 0x7f2ba009d690 2024-09-13T17:45:14.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.542+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f2b900b9940 con 0x7f2ba009d690 2024-09-13T17:45:14.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ba009bc00 con 0x7f2ba009d690 2024-09-13T17:45:14.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2b87fff640 1 -- 172.21.15.96:0/886167053 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f2ba009bc00 con 0x7f2ba009d690 2024-09-13T17:45:14.587 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:14.588 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:44:50.294936+0000","modified":"2024-09-13T17:44:53.298866+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5163},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5163":{"gid":5163,"name":"a","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/3293143360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3293143360},{"type":"v1","addr":"172.21.15.96:6829","nonce":3293143360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-09-13T17:45:14.588 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 msgr2=0x7f2b8c037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 0x7f2b8c037760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2ba0067370 tx=0x7f2b940079e0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 msgr2=0x7f2ba00a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2b9001c900 tx=0x7f2b9001ba70 comp rx=0 tx=0).stop 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 shutdown_connections 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2b8c0352a0 0x7f2b8c037760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 --2- 172.21.15.96:0/886167053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ba009d690 0x7f2ba00a6a60 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 >> 172.21.15.96:0/886167053 conn(0x7f2ba006bc60 msgr2=0x7f2ba0099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 shutdown_connections 2024-09-13T17:45:14.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.582+0000 7f2ba5db0640 1 -- 172.21.15.96:0/886167053 wait complete. 2024-09-13T17:45:14.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:14.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189f321640 1 Processor -- start 2024-09-13T17:45:14.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189f321640 1 -- start start 2024-09-13T17:45:14.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189f321640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f189809d690 0x7f1898138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189f321640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1898138050 con 0x7f18981387d0 2024-09-13T17:45:14.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189f321640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189809c410 con 0x7f189809d690 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189d0bd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f189809d690 0x7f1898138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189c8bc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f18981387d0 0x7f1898134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32770/0 (socket says 172.21.15.96:32770) 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189c8bc640 1 -- 172.21.15.96:0/3484977634 learned_addr learned my addr 172.21.15.96:0/3484977634 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189d0bd640 1 -- 172.21.15.96:0/3484977634 >> v1:172.21.15.96:6789/0 conn(0x7f18981387d0 legacy=0x7f1898134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f189d0bd640 1 -- 172.21.15.96:0/3484977634 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189809cba0 con 0x7f189809d690 2024-09-13T17:45:14.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.758+0000 7f188ffff640 1 -- 172.21.15.96:0/3484977634 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643309074 0 0) 0x7f1894001120 con 0x7f18981387d0 2024-09-13T17:45:14.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189d0bd640 1 --2- 172.21.15.96:0/3484977634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f1898138290 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1888009740 tx=0x7f188801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43ceae7c5d46bba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f188ffff640 1 -- 172.21.15.96:0/3484977634 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18880a8840 con 0x7f189809d690 2024-09-13T17:45:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f188ffff640 1 -- 172.21.15.96:0/3484977634 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18880a89a0 con 0x7f189809d690 2024-09-13T17:45:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f188ffff640 1 -- 172.21.15.96:0/3484977634 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18880a8c60 con 0x7f189809d690 2024-09-13T17:45:14.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- 172.21.15.96:0/3484977634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 msgr2=0x7f1898138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 --2- 172.21.15.96:0/3484977634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f1898138290 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1888009740 tx=0x7f188801ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- 172.21.15.96:0/3484977634 shutdown_connections 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 --2- 172.21.15.96:0/3484977634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f1898138290 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- 172.21.15.96:0/3484977634 >> 172.21.15.96:0/3484977634 conn(0x7f189806bc60 msgr2=0x7f189806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- 172.21.15.96:0/3484977634 shutdown_connections 2024-09-13T17:45:14.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- 172.21.15.96:0/3484977634 wait complete. 2024-09-13T17:45:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 Processor -- start 2024-09-13T17:45:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.762+0000 7f189f321640 1 -- start start 2024-09-13T17:45:14.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189f321640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189f321640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980a4360 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189d0bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189d0bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60514/0 (socket says 172.21.15.96:60514) 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189d0bd640 1 -- 172.21.15.96:0/4157983625 learned_addr learned my addr 172.21.15.96:0/4157983625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189d0bd640 1 -- 172.21.15.96:0/4157983625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1888009420 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189d0bd640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f188801c840 tx=0x7f188801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18880b0840 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18880b09a0 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980a7040 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18880b0c60 con 0x7f189809d690 2024-09-13T17:45:14.772 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980a5db0 con 0x7f189809d690 2024-09-13T17:45:14.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f18880b7040 con 0x7f189809d690 2024-09-13T17:45:14.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189809bc00 con 0x7f189809d690 2024-09-13T17:45:14.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 0x7f187c037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:14.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f18880e5820 con 0x7f189809d690 2024-09-13T17:45:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189c8bc640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 0x7f187c037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:14.774 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.766+0000 7f189c8bc640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 0x7f187c037950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c10 tx=0x7f1894001260 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:14.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.770+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f189809bc00 con 0x7f189809d690 2024-09-13T17:45:14.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.942+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f189809bc00 con 0x7f189809d690 2024-09-13T17:45:14.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.942+0000 7f188dffb640 1 -- 172.21.15.96:0/4157983625 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+11555 (secure 0 0 0) 0x7f189809bc00 con 0x7f189809d690 2024-09-13T17:45:14.949 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:14.949 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":58,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:11.531861+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:44:48.116392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.0299999713897705,"score_stable":1.0299999713897705,"optimal_score":1,"raw_score_acting":1.0299999713897705,"raw_score_stable":1.0299999713897705,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-13T17:44:49.284535+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/692121072":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2400367826":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2022112443":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6825/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6824/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1206016791":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3403848071":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:14.949 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:14.942+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:14.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 msgr2=0x7f187c037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 0x7f187c037950 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1894000c10 tx=0x7f1894001260 comp rx=0 tx=0).stop 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 msgr2=0x7f18980a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f188801c840 tx=0x7f188801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 shutdown_connections 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f187c035490 0x7f187c037950 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 --2- 172.21.15.96:0/4157983625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f189809d690 0x7f18980a6b00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 >> 172.21.15.96:0/4157983625 conn(0x7f189806bc60 msgr2=0x7f1898099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 shutdown_connections 2024-09-13T17:45:14.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:14.946+0000 7f189f321640 1 -- 172.21.15.96:0/4157983625 wait complete. 2024-09-13T17:45:14.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:45:15.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11c63d2640 1 Processor -- start 2024-09-13T17:45:15.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11c63d2640 1 -- start start 2024-09-13T17:45:15.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11c63d2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11c0136090 0x7f11c0136470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11c63d2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c00a79f0 con 0x7f11c01369b0 2024-09-13T17:45:15.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11c63d2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c006a650 con 0x7f11c0136090 2024-09-13T17:45:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11c0136090 0x7f11c0136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11c0136090 0x7f11c0136470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60516/0 (socket says 172.21.15.96:60516) 2024-09-13T17:45:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 -- 172.21.15.96:0/1545734075 learned_addr learned my addr 172.21.15.96:0/1545734075 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 -- 172.21.15.96:0/1545734075 >> v1:172.21.15.96:6789/0 conn(0x7f11c01369b0 legacy=0x7f11c009ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 -- 172.21.15.96:0/1545734075 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c009c3d0 con 0x7f11c0136090 2024-09-13T17:45:15.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11bffff640 1 --2- 172.21.15.96:0/1545734075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c0136470 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76ace342c68489a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11beffd640 1 -- 172.21.15.96:0/1545734075 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b80ab530 con 0x7f11c0136090 2024-09-13T17:45:15.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.082+0000 7f11beffd640 1 -- 172.21.15.96:0/1545734075 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80a7050 con 0x7f11c0136090 2024-09-13T17:45:15.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11beffd640 1 -- 172.21.15.96:0/1545734075 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b80ab7e0 con 0x7f11c0136090 2024-09-13T17:45:15.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/1545734075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 msgr2=0x7f11c0136470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 --2- 172.21.15.96:0/1545734075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c0136470 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f11b80097f0 tx=0x7f11b801c450 comp rx=0 tx=0).stop 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/1545734075 shutdown_connections 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 --2- 172.21.15.96:0/1545734075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c0136470 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/1545734075 >> 172.21.15.96:0/1545734075 conn(0x7f11c006e020 msgr2=0x7f11c0070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/1545734075 shutdown_connections 2024-09-13T17:45:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/1545734075 wait complete. 2024-09-13T17:45:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 Processor -- start 2024-09-13T17:45:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- start start 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c0121a10 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60522/0 (socket says 172.21.15.96:60522) 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bffff640 1 -- 172.21.15.96:0/532079011 learned_addr learned my addr 172.21.15.96:0/532079011 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bffff640 1 -- 172.21.15.96:0/532079011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b80094d0 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bffff640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f11b801a080 tx=0x7f11b801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b80b6420 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b80b6580 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c013b840 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11b80b6840 con 0x7f11c0136090 2024-09-13T17:45:15.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c013b280 con 0x7f11c0136090 2024-09-13T17:45:15.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.086+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f11b80b2040 con 0x7f11c0136090 2024-09-13T17:45:15.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.090+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c0136470 con 0x7f11c0136090 2024-09-13T17:45:15.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.090+0000 7f11bcff9640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 0x7f11a00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.090+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4170+0+0 (secure 0 0 0) 0x7f11b80e1e30 con 0x7f11c0136090 2024-09-13T17:45:15.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.090+0000 7f11bf7fe640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 0x7f11a00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.090+0000 7f11bf7fe640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 0x7f11a00379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000f70 tx=0x7f11a8007a40 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.094+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f11b80c7d20 con 0x7f11c0136090 2024-09-13T17:45:15.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.294+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11c013b840 con 0x7f11c0136090 2024-09-13T17:45:15.301 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:45:15.294+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:45:15.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11bcff9640 1 -- 172.21.15.96:0/532079011 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f11c013b840 con 0x7f11c0136090 2024-09-13T17:45:15.316 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:45:15.316 INFO:tasks.ceph.mds.a.smithi096.stderr: -2220> 2024-09-13T17:44:55.883+0000 7f207e835640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 msgr2=0x7f11a00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 0x7f11a00379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11a8000f70 tx=0x7f11a8007a40 comp rx=0 tx=0).stop 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 msgr2=0x7f11c011dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f11b801a080 tx=0x7f11b801ac90 comp rx=0 tx=0).stop 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 shutdown_connections 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f11a00354e0 0x7f11a00379a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 --2- 172.21.15.96:0/532079011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11c0136090 0x7f11c011dbc0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 >> 172.21.15.96:0/532079011 conn(0x7f11c006e020 msgr2=0x7f11c0137cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 shutdown_connections 2024-09-13T17:45:15.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.310+0000 7f11c63d2640 1 -- 172.21.15.96:0/532079011 wait complete. 2024-09-13T17:45:15.335 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:45:15.346 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5868dad640 1 Processor -- start 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5868dad640 1 -- start start 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5868dad640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f58640a1370 0x7f58640a1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5868dad640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586409eca0 con 0x7f58640a1c90 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5868dad640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586409d440 con 0x7f58640a1370 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f586259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f58640a1370 0x7f58640a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5861d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f58640a1c90 0x7f58641381e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32794/0 (socket says 172.21.15.96:32794) 2024-09-13T17:45:15.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f5861d9b640 1 -- 172.21.15.96:0/898275075 learned_addr learned my addr 172.21.15.96:0/898275075 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:15.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f586259c640 1 -- 172.21.15.96:0/898275075 >> v1:172.21.15.96:6789/0 conn(0x7f58640a1c90 legacy=0x7f58641381e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f586259c640 1 -- 172.21.15.96:0/898275075 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640a28b0 con 0x7f58640a1370 2024-09-13T17:45:15.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.438+0000 7f586159a640 1 -- 172.21.15.96:0/898275075 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450009937 0 0) 0x7f5854001120 con 0x7f58640a1c90 2024-09-13T17:45:15.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586259c640 1 --2- 172.21.15.96:0/898275075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f58640a1750 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5858009740 tx=0x7f585801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82249d3cdb0dba06 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586159a640 1 -- 172.21.15.96:0/898275075 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58580ab530 con 0x7f58640a1370 2024-09-13T17:45:15.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586159a640 1 -- 172.21.15.96:0/898275075 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58580a7050 con 0x7f58640a1370 2024-09-13T17:45:15.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586159a640 1 -- 172.21.15.96:0/898275075 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58580ab7e0 con 0x7f58640a1370 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- 172.21.15.96:0/898275075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 msgr2=0x7f58640a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 --2- 172.21.15.96:0/898275075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f58640a1750 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5858009740 tx=0x7f585801c390 comp rx=0 tx=0).stop 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- 172.21.15.96:0/898275075 shutdown_connections 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 --2- 172.21.15.96:0/898275075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f58640a1750 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- 172.21.15.96:0/898275075 >> 172.21.15.96:0/898275075 conn(0x7f58640a6b40 msgr2=0x7f58640a6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- 172.21.15.96:0/898275075 shutdown_connections 2024-09-13T17:45:15.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- 172.21.15.96:0/898275075 wait complete. 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 Processor -- start 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- start start 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f5868dad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864120920 con 0x7f58640a1370 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60580/0 (socket says 172.21.15.96:60580) 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.442+0000 7f586259c640 1 -- 172.21.15.96:0/329604593 learned_addr learned my addr 172.21.15.96:0/329604593 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:15.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f586259c640 1 -- 172.21.15.96:0/329604593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009420 con 0x7f58640a1370 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f586259c640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5858019fc0 tx=0x7f585801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58580b7450 con 0x7f58640a1370 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58580b75b0 con 0x7f58640a1370 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5864071a70 con 0x7f58640a1370 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58580b7870 con 0x7f58640a1370 2024-09-13T17:45:15.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586411d480 con 0x7f58640a1370 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5864065ee0 con 0x7f58640a1370 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f58580b79d0 con 0x7f58640a1370 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 0x7f58440379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4256+0+0 (secure 0 0 0) 0x7f58580e24f0 con 0x7f58640a1370 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5861d9b640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 0x7f58440379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:15.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.446+0000 7f5861d9b640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 0x7f58440379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5854005890 tx=0x7f5854001280 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:15.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.450+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f58580c74e0 con 0x7f58640a1370 2024-09-13T17:45:15.637 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:15.630+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:15.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:15.650+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f586411ea30 con 0x7f58640a1370 2024-09-13T17:45:16.315 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.310+0000 7f5852ffd640 1 -- 172.21.15.96:0/329604593 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f58580bb0a0 con 0x7f58640a1370 2024-09-13T17:45:16.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 msgr2=0x7f58440379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:16.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 0x7f58440379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5854005890 tx=0x7f5854001280 comp rx=0 tx=0).stop 2024-09-13T17:45:16.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 msgr2=0x7f5864121000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:16.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f5858019fc0 tx=0x7f585801abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:16.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 shutdown_connections 2024-09-13T17:45:16.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f58440354e0 0x7f58440379a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:16.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 --2- 172.21.15.96:0/329604593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58640a1370 0x7f5864121000 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:16.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 >> 172.21.15.96:0/329604593 conn(0x7f58640a6b40 msgr2=0x7f586406dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:16.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 shutdown_connections 2024-09-13T17:45:16.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.314+0000 7f5868dad640 1 -- 172.21.15.96:0/329604593 wait complete. 2024-09-13T17:45:16.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:45:16.430 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:16.422+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:16.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f752a0b4640 1 Processor -- start 2024-09-13T17:45:16.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f752a0b4640 1 -- start start 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f752a0b4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f752412de90 0x7f752412e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f752a0b4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752406fe50 con 0x7f752406e5c0 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f752a0b4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75240707d0 con 0x7f752412de90 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f7522ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f752406e5c0 0x7f752406ea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32798/0 (socket says 172.21.15.96:32798) 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f75237fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f752412de90 0x7f752412e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f7522ffd640 1 -- 172.21.15.96:0/1109237771 learned_addr learned my addr 172.21.15.96:0/1109237771 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f75227fc640 1 -- 172.21.15.96:0/1109237771 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344057829 0 0) 0x7f752406fe50 con 0x7f752406e5c0 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f75227fc640 1 -- 172.21.15.96:0/1109237771 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f750c003530 con 0x7f752406e5c0 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f75237fe640 1 -- 172.21.15.96:0/1109237771 >> v1:172.21.15.96:6789/0 conn(0x7f752406e5c0 legacy=0x7f752406ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:16.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.514+0000 7f75237fe640 1 -- 172.21.15.96:0/1109237771 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524071650 con 0x7f752412de90 2024-09-13T17:45:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 --2- 172.21.15.96:0/1109237771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752412de90 0x7f752412e270 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7514009740 tx=0x7f751401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a4c5b886d1ee1b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75227fc640 1 -- 172.21.15.96:0/1109237771 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75140ab530 con 0x7f752412de90 2024-09-13T17:45:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75227fc640 1 -- 172.21.15.96:0/1109237771 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75140a7050 con 0x7f752412de90 2024-09-13T17:45:16.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75227fc640 1 -- 172.21.15.96:0/1109237771 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75140ab7e0 con 0x7f752412de90 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/1109237771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752412de90 msgr2=0x7f752412e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 --2- 172.21.15.96:0/1109237771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752412de90 0x7f752412e270 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f7514009740 tx=0x7f751401c360 comp rx=0 tx=0).stop 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/1109237771 shutdown_connections 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 --2- 172.21.15.96:0/1109237771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752412de90 0x7f752412e270 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/1109237771 >> 172.21.15.96:0/1109237771 conn(0x7f7524098a60 msgr2=0x7f752409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:16.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/1109237771 shutdown_connections 2024-09-13T17:45:16.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/1109237771 wait complete. 2024-09-13T17:45:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 Processor -- start 2024-09-13T17:45:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- start start 2024-09-13T17:45:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:16.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7524133d70 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60590/0 (socket says 172.21.15.96:60590) 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 -- 172.21.15.96:0/172957303 learned_addr learned my addr 172.21.15.96:0/172957303 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 -- 172.21.15.96:0/172957303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7514009420 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f75237fe640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7514019f90 tx=0x7f751401aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75140abe10 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75140b7450 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7524133fc0 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75140b76f0 con 0x7f752406e5c0 2024-09-13T17:45:16.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.518+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752413c2a0 con 0x7f752406e5c0 2024-09-13T17:45:16.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f75140b7850 con 0x7f752406e5c0 2024-09-13T17:45:16.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7524131b50 con 0x7f752406e5c0 2024-09-13T17:45:16.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f750bfff640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 0x7f75100379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:16.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 4256+0+0 (secure 0 0 0) 0x7f75140e2570 con 0x7f752406e5c0 2024-09-13T17:45:16.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f7522ffd640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 0x7f75100379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:16.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.522+0000 7f7522ffd640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 0x7f75100379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7518007fd0 tx=0x7f7518007980 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:16.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.526+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f75140c76c0 con 0x7f752406e5c0 2024-09-13T17:45:16.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:16.694+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f752409d540 con 0x7f752406e5c0 2024-09-13T17:45:17.216 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:17.210+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:17.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.318+0000 7f750bfff640 1 -- 172.21.15.96:0/172957303 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) v1 ==== 181+0+0 (secure 0 0 0) 0x7f75140b0020 con 0x7f752406e5c0 2024-09-13T17:45:17.325 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 msgr2=0x7f75100379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 0x7f75100379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7518007fd0 tx=0x7f7518007980 comp rx=0 tx=0).stop 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 msgr2=0x7f7524132af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7514019f90 tx=0x7f751401aba0 comp rx=0 tx=0).stop 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 shutdown_connections 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f75100354e0 0x7f75100379a0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7518007fd0 tx=0x7f7518007980 comp rx=0 tx=0).stop 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 --2- 172.21.15.96:0/172957303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f752406e5c0 0x7f7524132af0 secure :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f7514019f90 tx=0x7f751401aba0 comp rx=0 tx=0).stop 2024-09-13T17:45:17.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 >> 172.21.15.96:0/172957303 conn(0x7f7524098a60 msgr2=0x7f7524099cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:17.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 shutdown_connections 2024-09-13T17:45:17.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.326+0000 7f752a0b4640 1 -- 172.21.15.96:0/172957303 wait complete. 2024-09-13T17:45:17.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:45:17.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f5417c62640 1 Processor -- start 2024-09-13T17:45:17.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f5417c62640 1 -- start start 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f5417c62640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5410137460 0x7f5410139850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f5417c62640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f541006fe50 con 0x7f541012de90 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f5417c62640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54100707d0 con 0x7f5410137460 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54151fd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5410137460 0x7f5410139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54159fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f541012de90 0x7f541012e270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32812/0 (socket says 172.21.15.96:32812) 2024-09-13T17:45:17.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54159fe640 1 -- 172.21.15.96:0/3866124928 learned_addr learned my addr 172.21.15.96:0/3866124928 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:17.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54151fd640 1 -- 172.21.15.96:0/3866124928 >> v1:172.21.15.96:6789/0 conn(0x7f541012de90 legacy=0x7f541012e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:17.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54151fd640 1 -- 172.21.15.96:0/3866124928 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410071650 con 0x7f5410137460 2024-09-13T17:45:17.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.482+0000 7f54149fc640 1 -- 172.21.15.96:0/3866124928 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940031860 0 0) 0x7f541006fe50 con 0x7f541012de90 2024-09-13T17:45:17.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54151fd640 1 --2- 172.21.15.96:0/3866124928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5410137460 0x7f5410139850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f540c009740 tx=0x7f540c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e951c352c8996ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:17.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54149fc640 1 -- 172.21.15.96:0/3866124928 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f540c0ad5b0 con 0x7f5410137460 2024-09-13T17:45:17.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54149fc640 1 -- 172.21.15.96:0/3866124928 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c0ad710 con 0x7f5410137460 2024-09-13T17:45:17.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54149fc640 1 -- 172.21.15.96:0/3866124928 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f540c0ad9d0 con 0x7f5410137460 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/3866124928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5410137460 msgr2=0x7f5410139850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 --2- 172.21.15.96:0/3866124928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5410137460 0x7f5410139850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f540c009740 tx=0x7f540c01c360 comp rx=0 tx=0).stop 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/3866124928 shutdown_connections 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 --2- 172.21.15.96:0/3866124928 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5410137460 0x7f5410139850 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/3866124928 >> 172.21.15.96:0/3866124928 conn(0x7f5410098a60 msgr2=0x7f541009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/3866124928 shutdown_connections 2024-09-13T17:45:17.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/3866124928 wait complete. 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 Processor -- start 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- start start 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54100a3e00 con 0x7f541012de90 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54159fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54159fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60596/0 (socket says 172.21.15.96:60596) 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54159fe640 1 -- 172.21.15.96:0/1242217044 learned_addr learned my addr 172.21.15.96:0/1242217044 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54159fe640 1 -- 172.21.15.96:0/1242217044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c009420 con 0x7f541012de90 2024-09-13T17:45:17.494 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54159fe640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f540000c390 tx=0x7f540000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5400007410 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5400007590 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54101397c0 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54000047f0 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.486+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54101394e0 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f541012e270 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5400004950 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f54067fc640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 0x7f53f40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 3818+0+0 (secure 0 0 0) 0x7f5400047600 con 0x7f541012de90 2024-09-13T17:45:17.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f54151fd640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 0x7f53f40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:17.497 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.490+0000 7f54151fd640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 0x7f53f40379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f540c0af000 tx=0x7f540c008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:17.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.494+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f540001bb70 con 0x7f541012de90 2024-09-13T17:45:17.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:17.662+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f541006a1b0 con 0x7f541012de90 2024-09-13T17:45:18.008 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:18.002+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:18.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.326+0000 7f54067fc640 1 -- 172.21.15.96:0/1242217044 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) v1 ==== 169+0+0 (secure 0 0 0) 0x7f540001b340 con 0x7f541012de90 2024-09-13T17:45:18.336 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 msgr2=0x7f53f40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 0x7f53f40379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f540c0af000 tx=0x7f540c008040 comp rx=0 tx=0).stop 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 msgr2=0x7f541012ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f540000c390 tx=0x7f540000c850 comp rx=0 tx=0).stop 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 shutdown_connections 2024-09-13T17:45:18.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f53f40354e0 0x7f53f40379a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 --2- 172.21.15.96:0/1242217044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f541012de90 0x7f541012ca30 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 >> 172.21.15.96:0/1242217044 conn(0x7f5410098a60 msgr2=0x7f541006e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 shutdown_connections 2024-09-13T17:45:18.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.330+0000 7f5417c62640 1 -- 172.21.15.96:0/1242217044 wait complete. 2024-09-13T17:45:18.362 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:45:18.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56f557c640 1 Processor -- start 2024-09-13T17:45:18.771 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56f557c640 1 -- start start 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56f557c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f56f01357b0 0x7f56f0139e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56f557c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0134af0 con 0x7f56f009b0a0 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56f557c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f009b480 con 0x7f56f01357b0 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56ee7fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f56f01357b0 0x7f56f0139e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56eeffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f56f009b0a0 0x7f56f01350a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32814/0 (socket says 172.21.15.96:32814) 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56eeffd640 1 -- 172.21.15.96:0/3338245683 learned_addr learned my addr 172.21.15.96:0/3338245683 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:18.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56ee7fc640 1 -- 172.21.15.96:0/3338245683 >> v1:172.21.15.96:6789/0 conn(0x7f56f009b0a0 legacy=0x7f56f01350a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.482+0000 7f56ee7fc640 1 -- 172.21.15.96:0/3338245683 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f009ccc0 con 0x7f56f01357b0 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56ee7fc640 1 --2- 172.21.15.96:0/3338245683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f01357b0 0x7f56f0139e70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009740 tx=0x7f56d801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b78ef5c139a896 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56edffb640 1 -- 172.21.15.96:0/3338245683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56d80ab530 con 0x7f56f01357b0 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56edffb640 1 -- 172.21.15.96:0/3338245683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d80a7050 con 0x7f56f01357b0 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56edffb640 1 -- 172.21.15.96:0/3338245683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56d80ab7e0 con 0x7f56f01357b0 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 -- 172.21.15.96:0/3338245683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f01357b0 msgr2=0x7f56f0139e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 --2- 172.21.15.96:0/3338245683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f01357b0 0x7f56f0139e70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009740 tx=0x7f56d801c330 comp rx=0 tx=0).stop 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 -- 172.21.15.96:0/3338245683 shutdown_connections 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 --2- 172.21.15.96:0/3338245683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f01357b0 0x7f56f0139e70 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 -- 172.21.15.96:0/3338245683 >> 172.21.15.96:0/3338245683 conn(0x7f56f006e020 msgr2=0x7f56f0070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 -- 172.21.15.96:0/3338245683 shutdown_connections 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.486+0000 7f56f557c640 1 -- 172.21.15.96:0/3338245683 wait complete. 2024-09-13T17:45:18.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 Processor -- start 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 -- start start 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f00a06f0 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56eeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56eeffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60614/0 (socket says 172.21.15.96:60614) 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56eeffd640 1 -- 172.21.15.96:0/3441509596 learned_addr learned my addr 172.21.15.96:0/3441509596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56eeffd640 1 -- 172.21.15.96:0/3441509596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d8009420 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56eeffd640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c380 tx=0x7f56e800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56e8005c20 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e8005d80 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56e8007200 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f012cce0 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f00a0430 con 0x7f56f009b0a0 2024-09-13T17:45:18.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f56e8007360 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 0x7f56d00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7f56e8046be0 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f00a0830 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56ee7fc640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 0x7f56d00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.490+0000 7f56ee7fc640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 0x7f56d00379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ad000 tx=0x7f56d8008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.494+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f56f00a0830 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.662+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f56f00a0830 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56e77fe640 1 -- 172.21.15.96:0/3441509596 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56f00a0830 con 0x7f56f009b0a0 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 msgr2=0x7f56d00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 0x7f56d00379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f56d80ad000 tx=0x7f56d8008040 comp rx=0 tx=0).stop 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 msgr2=0x7f56f009e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f56e800c380 tx=0x7f56e800c840 comp rx=0 tx=0).stop 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 shutdown_connections 2024-09-13T17:45:18.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f56d00354e0 0x7f56d00379a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 --2- 172.21.15.96:0/3441509596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56f009b0a0 0x7f56f009e1a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.666+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 >> 172.21.15.96:0/3441509596 conn(0x7f56f006e020 msgr2=0x7f56f0133880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:18.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.670+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 shutdown_connections 2024-09-13T17:45:18.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.670+0000 7f56f557c640 1 -- 172.21.15.96:0/3441509596 wait complete. 2024-09-13T17:45:18.780 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop' 2024-09-13T17:45:18.794 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:18.786+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:18.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.906+0000 7f2ff4f8b640 1 Processor -- start 2024-09-13T17:45:18.915 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2ff4f8b640 1 -- start start 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2ff4f8b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ff0138080 0x7f2ff0138460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2ff4f8b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00a2290 con 0x7f2ff01389a0 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2ff4f8b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff006a8d0 con 0x7f2ff0138080 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fee59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2ff0138080 0x7f2ff0138460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fe5d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2ff01389a0 0x7f2ff009ce70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32816/0 (socket says 172.21.15.96:32816) 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fe5d9b640 1 -- 172.21.15.96:0/1463133230 learned_addr learned my addr 172.21.15.96:0/1463133230 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:18.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fee59c640 1 -- 172.21.15.96:0/1463133230 >> v1:172.21.15.96:6789/0 conn(0x7f2ff01389a0 legacy=0x7f2ff009ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fee59c640 1 -- 172.21.15.96:0/1463133230 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00a2e00 con 0x7f2ff0138080 2024-09-13T17:45:18.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fee59c640 1 --2- 172.21.15.96:0/1463133230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff0138460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c901d5618abbf15b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fedd9b640 1 -- 172.21.15.96:0/1463133230 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fe00a56d0 con 0x7f2ff0138080 2024-09-13T17:45:18.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fedd9b640 1 -- 172.21.15.96:0/1463133230 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00a5830 con 0x7f2ff0138080 2024-09-13T17:45:18.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.910+0000 7f2fedd9b640 1 -- 172.21.15.96:0/1463133230 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fe00a5af0 con 0x7f2ff0138080 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/1463133230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 msgr2=0x7f2ff0138460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 --2- 172.21.15.96:0/1463133230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff0138460 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009d90 tx=0x7f2fe001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/1463133230 shutdown_connections 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 --2- 172.21.15.96:0/1463133230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff0138460 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/1463133230 >> 172.21.15.96:0/1463133230 conn(0x7f2ff00a7390 msgr2=0x7f2ff00a77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/1463133230 shutdown_connections 2024-09-13T17:45:18.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/1463133230 wait complete. 2024-09-13T17:45:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 Processor -- start 2024-09-13T17:45:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- start start 2024-09-13T17:45:18.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0071b70 con 0x7f2ff0138080 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fee59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fee59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60622/0 (socket says 172.21.15.96:60622) 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fee59c640 1 -- 172.21.15.96:0/3967496493 learned_addr learned my addr 172.21.15.96:0/3967496493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fee59c640 1 -- 172.21.15.96:0/3967496493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009a70 con 0x7f2ff0138080 2024-09-13T17:45:18.921 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fee59c640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2fe001a510 tx=0x7f2fe0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fe00ad800 con 0x7f2ff0138080 2024-09-13T17:45:18.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe00ad960 con 0x7f2ff0138080 2024-09-13T17:45:18.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/3967496493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff0071d60 con 0x7f2ff0138080 2024-09-13T17:45:18.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2fe00adc40 con 0x7f2ff0138080 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2ff4f8b640 1 -- 172.21.15.96:0/3967496493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff006f7f0 con 0x7f2ff0138080 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc8005260 con 0x7f2ff0138080 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f2fe00b4050 con 0x7f2ff0138080 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 0x7f2fd00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7f2fe00e2a20 con 0x7f2ff0138080 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.914+0000 7f2fe5d9b640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 0x7f2fd00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:18.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.918+0000 7f2fe5d9b640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 0x7f2fd00379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8005890 tx=0x7f2fd8007a90 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:18.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:18.922+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2fe00c0b90 con 0x7f2ff0138080 2024-09-13T17:45:19.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.162+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop"]} v 0) v1 -- 0x7f2fc80057e0 con 0x7f2ff0138080 2024-09-13T17:45:19.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.322+0000 7f2fe77fe640 1 -- 172.21.15.96:0/3967496493 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop"]}]=0 v197) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2fe00b2030 con 0x7f2ff0138080 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 msgr2=0x7f2fd00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 0x7f2fd00379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8005890 tx=0x7f2fd8007a90 comp rx=0 tx=0).stop 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 msgr2=0x7f2ff006ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f2fe001a510 tx=0x7f2fe0019680 comp rx=0 tx=0).stop 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 shutdown_connections 2024-09-13T17:45:19.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f2fd00354e0 0x7f2fd00379a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 --2- 172.21.15.96:0/3967496493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ff0138080 0x7f2ff006ec10 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 >> 172.21.15.96:0/3967496493 conn(0x7f2ff00a7390 msgr2=0x7f2ff0137290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:19.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 shutdown_connections 2024-09-13T17:45:19.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.326+0000 7f2fe4d99640 1 -- 172.21.15.96:0/3967496493 wait complete. 2024-09-13T17:45:19.354 INFO:tasks.cephfs_test_runner:test_fstop (tasks.cephfs.test_fstop.TestFSTop) ... ok 2024-09-13T17:45:19.354 INFO:tasks.cephfs_test_runner:Starting test: test_fstop_non_existent_cluster (tasks.cephfs.test_fstop.TestFSTop) 2024-09-13T17:45:19.354 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ef2a640 1 Processor -- start 2024-09-13T17:45:19.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ef2a640 1 -- start start 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ef2a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9e58134670 0x7f9e58134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ef2a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e58071670 con 0x7f9e5806e5c0 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ef2a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e58071360 con 0x7f9e58134670 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ccc6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9e58134670 0x7f9e58134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e57fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9e5806e5c0 0x7f9e5806ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32820/0 (socket says 172.21.15.96:32820) 2024-09-13T17:45:19.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e57fff640 1 -- 172.21.15.96:0/3388614277 learned_addr learned my addr 172.21.15.96:0/3388614277 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ccc6640 1 -- 172.21.15.96:0/3388614277 >> v1:172.21.15.96:6789/0 conn(0x7f9e5806e5c0 legacy=0x7f9e5806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e5ccc6640 1 -- 172.21.15.96:0/3388614277 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580a7200 con 0x7f9e58134670 2024-09-13T17:45:19.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.514+0000 7f9e577fe640 1 -- 172.21.15.96:0/3388614277 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791515107 0 0) 0x7f9e58071670 con 0x7f9e5806e5c0 2024-09-13T17:45:19.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ccc6640 1 --2- 172.21.15.96:0/3388614277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e58134670 0x7f9e58134a50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000aea0 tx=0x7f9e5001d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3636bfea30093432 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e577fe640 1 -- 172.21.15.96:0/3388614277 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e5000a200 con 0x7f9e58134670 2024-09-13T17:45:19.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e577fe640 1 -- 172.21.15.96:0/3388614277 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500a66d0 con 0x7f9e58134670 2024-09-13T17:45:19.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e577fe640 1 -- 172.21.15.96:0/3388614277 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e500a6990 con 0x7f9e58134670 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3388614277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e58134670 msgr2=0x7f9e58134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3388614277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e58134670 0x7f9e58134a50 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000aea0 tx=0x7f9e5001d9e0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3388614277 shutdown_connections 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3388614277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e58134670 0x7f9e58134a50 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3388614277 >> 172.21.15.96:0/3388614277 conn(0x7f9e58098a60 msgr2=0x7f9e5809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3388614277 shutdown_connections 2024-09-13T17:45:19.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3388614277 wait complete. 2024-09-13T17:45:19.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 Processor -- start 2024-09-13T17:45:19.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.518+0000 7f9e5ef2a640 1 -- start start 2024-09-13T17:45:19.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ef2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ef2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5809a3d0 con 0x7f9e5806e5c0 2024-09-13T17:45:19.527 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ccc6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ccc6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60644/0 (socket says 172.21.15.96:60644) 2024-09-13T17:45:19.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ccc6640 1 -- 172.21.15.96:0/3037451214 learned_addr learned my addr 172.21.15.96:0/3037451214 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:19.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ccc6640 1 -- 172.21.15.96:0/3037451214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5000ab80 con 0x7f9e5806e5c0 2024-09-13T17:45:19.528 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ccc6640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a5f0 tx=0x7f9e5001a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e500b7780 con 0x7f9e5806e5c0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e500b78e0 con 0x7f9e5806e5c0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e580a44c0 con 0x7f9e5806e5c0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9e500b7bc0 con 0x7f9e5806e5c0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5809ae10 con 0x7f9e5806e5c0 2024-09-13T17:45:19.529 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f9e500b7d20 con 0x7f9e5806e5c0 2024-09-13T17:45:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5809d520 con 0x7f9e5806e5c0 2024-09-13T17:45:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 0x7f9e440379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7f9e500e2e60 con 0x7f9e5806e5c0 2024-09-13T17:45:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e57fff640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 0x7f9e440379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.522+0000 7f9e57fff640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 0x7f9e440379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c007fd0 tx=0x7f9e4c007920 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.526+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e500b5340 con 0x7f9e5806e5c0 2024-09-13T17:45:19.559 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:19.554+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:19.703 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.698+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e5806a1b0 con 0x7f9e5806e5c0 2024-09-13T17:45:19.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.698+0000 7f9e557fa640 1 -- 172.21.15.96:0/3037451214 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+7915 (secure 0 0 0) 0x7f9e500bc020 con 0x7f9e5806e5c0 2024-09-13T17:45:19.704 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:19.704 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":61,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:18.318140+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6828/3293143360":"2024-09-14T17:45:15.300879+0000","172.21.15.96:0/692121072":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2400367826":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2022112443":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6825/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:6824/420999973":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2016081916":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6829/3293143360":"2024-09-14T17:45:15.300879+0000","172.21.15.96:0/3931971068":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1206016791":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/2331602584":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6825/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:6824/3506364525":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/2147498054":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1429934372":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/1435736606":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/3403848071":"2024-09-14T17:45:11.531783+0000","172.21.15.96:0/4091393434":"2024-09-14T17:45:06.004232+0000","172.21.15.96:0/799438860":"2024-09-14T17:45:06.004232+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:19.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 msgr2=0x7f9e440379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 0x7f9e440379a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c007fd0 tx=0x7f9e4c007920 comp rx=0 tx=0).stop 2024-09-13T17:45:19.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 msgr2=0x7f9e581cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9e5001a5f0 tx=0x7f9e5001a780 comp rx=0 tx=0).stop 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 shutdown_connections 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9e440354e0 0x7f9e440379a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 --2- 172.21.15.96:0/3037451214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e5806e5c0 0x7f9e581cc910 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 >> 172.21.15.96:0/3037451214 conn(0x7f9e58098a60 msgr2=0x7f9e581c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 shutdown_connections 2024-09-13T17:45:19.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.702+0000 7f9e5ef2a640 1 -- 172.21.15.96:0/3037451214 wait complete. 2024-09-13T17:45:19.724 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:19.725 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:19.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f456a71c640 1 Processor -- start 2024-09-13T17:45:19.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f456a71c640 1 -- start start 2024-09-13T17:45:19.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f456a71c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4564136020 0x7f456409e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f456a71c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456406ad30 con 0x7f4564136400 2024-09-13T17:45:19.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f456a71c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456409ef90 con 0x7f4564136020 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f4563fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f4564136020 0x7f456409e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f455b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f4564136400 0x7f456409ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32834/0 (socket says 172.21.15.96:32834) 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f455b7fe640 1 -- 172.21.15.96:0/13237760 learned_addr learned my addr 172.21.15.96:0/13237760 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f4563fff640 1 -- 172.21.15.96:0/13237760 >> v1:172.21.15.96:6789/0 conn(0x7f4564136400 legacy=0x7f456409ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f45637fe640 1 -- 172.21.15.96:0/13237760 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410170327 0 0) 0x7f456406ad30 con 0x7f4564136400 2024-09-13T17:45:19.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f4563fff640 1 -- 172.21.15.96:0/13237760 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456406a860 con 0x7f4564136020 2024-09-13T17:45:19.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f4563fff640 1 --2- 172.21.15.96:0/13237760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f456409e990 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4554009740 tx=0x7f455401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68948ac4bdf45dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f45637fe640 1 -- 172.21.15.96:0/13237760 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45540ab400 con 0x7f4564136020 2024-09-13T17:45:19.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f45637fe640 1 -- 172.21.15.96:0/13237760 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45540a7050 con 0x7f4564136020 2024-09-13T17:45:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.850+0000 7f45637fe640 1 -- 172.21.15.96:0/13237760 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45540ab6b0 con 0x7f4564136020 2024-09-13T17:45:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/13237760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 msgr2=0x7f456409e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 --2- 172.21.15.96:0/13237760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f456409e990 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f4554009740 tx=0x7f455401c3e0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/13237760 shutdown_connections 2024-09-13T17:45:19.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 --2- 172.21.15.96:0/13237760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f456409e990 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:19.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/13237760 >> 172.21.15.96:0/13237760 conn(0x7f456406e350 msgr2=0x7f4564070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:19.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/13237760 shutdown_connections 2024-09-13T17:45:19.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/13237760 wait complete. 2024-09-13T17:45:19.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 Processor -- start 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- start start 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45640a3a60 con 0x7f4564136020 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f4563fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f4563fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60662/0 (socket says 172.21.15.96:60662) 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f4563fff640 1 -- 172.21.15.96:0/2334467924 learned_addr learned my addr 172.21.15.96:0/2334467924 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f4563fff640 1 -- 172.21.15.96:0/2334467924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4554009420 con 0x7f4564136020 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f4563fff640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f455401a010 tx=0x7f455401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4554004020 con 0x7f4564136020 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554004180 con 0x7f4564136020 2024-09-13T17:45:19.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45640a4cf0 con 0x7f4564136020 2024-09-13T17:45:19.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4554004460 con 0x7f4564136020 2024-09-13T17:45:19.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.854+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45640a0670 con 0x7f4564136020 2024-09-13T17:45:19.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f4554003620 con 0x7f4564136020 2024-09-13T17:45:19.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640a2270 con 0x7f4564136020 2024-09-13T17:45:19.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f45617fa640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 0x7f45440379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:19.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7f45540e1f50 con 0x7f4564136020 2024-09-13T17:45:19.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f455b7fe640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 0x7f45440379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:19.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.858+0000 7f455b7fe640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 0x7f45440379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f454c007fd0 tx=0x7f454c007980 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:19.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:19.862+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f45640a2270 con 0x7f4564136020 2024-09-13T17:45:20.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45640a2270 con 0x7f4564136020 2024-09-13T17:45:20.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f45617fa640 1 -- 172.21.15.96:0/2334467924 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45640a2270 con 0x7f4564136020 2024-09-13T17:45:20.036 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 msgr2=0x7f45440379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f456a71c640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 0x7f45440379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f454c007fd0 tx=0x7f454c007980 comp rx=0 tx=0).stop 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 msgr2=0x7f45640a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.030+0000 7f456a71c640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f455401a010 tx=0x7f455401ac20 comp rx=0 tx=0).stop 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 shutdown_connections 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f45440354e0 0x7f45440379a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 --2- 172.21.15.96:0/2334467924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4564136020 0x7f45640a47b0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 >> 172.21.15.96:0/2334467924 conn(0x7f456406e350 msgr2=0x7f45641351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 shutdown_connections 2024-09-13T17:45:20.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.034+0000 7f456a71c640 1 -- 172.21.15.96:0/2334467924 wait complete. 2024-09-13T17:45:20.057 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster' 2024-09-13T17:45:20.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f3074ed3640 1 Processor -- start 2024-09-13T17:45:20.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f3074ed3640 1 -- start start 2024-09-13T17:45:20.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f3074ed3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f307009bb10 0x7f307013d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:20.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f3074ed3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30700a7b90 con 0x7f3070137210 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f3074ed3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307006a7a0 con 0x7f307009bb10 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306dd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f307009bb10 0x7f307013d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306e59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3070137210 0x7f30701375f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32844/0 (socket says 172.21.15.96:32844) 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306e59c640 1 -- 172.21.15.96:0/3234073530 learned_addr learned my addr 172.21.15.96:0/3234073530 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306dd9b640 1 -- 172.21.15.96:0/3234073530 >> v1:172.21.15.96:6789/0 conn(0x7f3070137210 legacy=0x7f30701375f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306dd9b640 1 -- 172.21.15.96:0/3234073530 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307009ae70 con 0x7f307009bb10 2024-09-13T17:45:20.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.178+0000 7f306d59a640 1 -- 172.21.15.96:0/3234073530 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251527536 0 0) 0x7f30700a7b90 con 0x7f3070137210 2024-09-13T17:45:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306dd9b640 1 --2- 172.21.15.96:0/3234073530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307013d160 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3064009740 tx=0x7f306401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c5496e679c281f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306d59a640 1 -- 172.21.15.96:0/3234073530 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30640ab400 con 0x7f307009bb10 2024-09-13T17:45:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306d59a640 1 -- 172.21.15.96:0/3234073530 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30640a7050 con 0x7f307009bb10 2024-09-13T17:45:20.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306d59a640 1 -- 172.21.15.96:0/3234073530 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f30640ab6b0 con 0x7f307009bb10 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/3234073530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 msgr2=0x7f307013d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 --2- 172.21.15.96:0/3234073530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307013d160 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f3064009740 tx=0x7f306401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/3234073530 shutdown_connections 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 --2- 172.21.15.96:0/3234073530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307013d160 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/3234073530 >> 172.21.15.96:0/3234073530 conn(0x7f3070066870 msgr2=0x7f307006ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/3234073530 shutdown_connections 2024-09-13T17:45:20.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/3234073530 wait complete. 2024-09-13T17:45:20.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 Processor -- start 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- start start 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700a5ce0 con 0x7f307009bb10 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60672/0 (socket says 172.21.15.96:60672) 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306e59c640 1 -- 172.21.15.96:0/745951816 learned_addr learned my addr 172.21.15.96:0/745951816 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306e59c640 1 -- 172.21.15.96:0/745951816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064009420 con 0x7f307009bb10 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f306e59c640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f306000cb80 tx=0x7f306000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f306000ac90 con 0x7f307009bb10 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000adf0 con 0x7f307009bb10 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3060012530 con 0x7f307009bb10 2024-09-13T17:45:20.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700a6da0 con 0x7f307009bb10 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.182+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30700a5fb0 con 0x7f307009bb10 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f3060012690 con 0x7f307009bb10 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700a3a40 con 0x7f307009bb10 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f305effd640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 0x7f30500379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7f3060047cc0 con 0x7f307009bb10 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f306dd9b640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 0x7f30500379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:20.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.186+0000 7f306dd9b640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 0x7f30500379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f30640ad000 tx=0x7f3064008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:20.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.190+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3060019300 con 0x7f307009bb10 2024-09-13T17:45:20.295 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:20.286+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:20.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:20.418+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster"]} v 0) v1 -- 0x7f30700a5400 con 0x7f307009bb10 2024-09-13T17:45:21.159 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:21.150+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:21.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.326+0000 7f305effd640 1 -- 172.21.15.96:0/745951816 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster"]}]=0 v199) v1 ==== 147+0+0 (secure 0 0 0) 0x7f3060010af0 con 0x7f307009bb10 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 msgr2=0x7f30500379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 0x7f30500379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f30640ad000 tx=0x7f3064008040 comp rx=0 tx=0).stop 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 msgr2=0x7f307011dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f306000cb80 tx=0x7f306000a430 comp rx=0 tx=0).stop 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 shutdown_connections 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f30500354e0 0x7f30500379a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 --2- 172.21.15.96:0/745951816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f307009bb10 0x7f307011dc30 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 >> 172.21.15.96:0/745951816 conn(0x7f3070066870 msgr2=0x7f307006f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 shutdown_connections 2024-09-13T17:45:21.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.330+0000 7f3074ed3640 1 -- 172.21.15.96:0/745951816 wait complete. 2024-09-13T17:45:21.358 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.474+0000 7fdef09e5640 1 Processor -- start 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.474+0000 7fdef09e5640 1 -- start start 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdeec134e70 0x7fdeec09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeec09ed60 con 0x7fdeec135250 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeec09ddc0 con 0x7fdeec134e70 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdeea59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdeec134e70 0x7fdeec09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee9d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdeec135250 0x7fdeec135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32848/0 (socket says 172.21.15.96:32848) 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee9d9b640 1 -- 172.21.15.96:0/2406893039 learned_addr learned my addr 172.21.15.96:0/2406893039 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdeea59c640 1 -- 172.21.15.96:0/2406893039 >> v1:172.21.15.96:6789/0 conn(0x7fdeec135250 legacy=0x7fdeec135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdeea59c640 1 -- 172.21.15.96:0/2406893039 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0a68b0 con 0x7fdeec134e70 2024-09-13T17:45:21.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee959a640 1 -- 172.21.15.96:0/2406893039 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856114551 0 0) 0x7fdedc0014a0 con 0x7fdeec135250 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdeea59c640 1 --2- 172.21.15.96:0/2406893039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec09e820 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009740 tx=0x7fdee401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace1854d60736c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee959a640 1 -- 172.21.15.96:0/2406893039 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee40ab530 con 0x7fdeec134e70 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee959a640 1 -- 172.21.15.96:0/2406893039 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee40a7050 con 0x7fdeec134e70 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdee959a640 1 -- 172.21.15.96:0/2406893039 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee40ab7e0 con 0x7fdeec134e70 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 -- 172.21.15.96:0/2406893039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 msgr2=0x7fdeec09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 --2- 172.21.15.96:0/2406893039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec09e820 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009740 tx=0x7fdee401c420 comp rx=0 tx=0).stop 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 -- 172.21.15.96:0/2406893039 shutdown_connections 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 --2- 172.21.15.96:0/2406893039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec09e820 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.478+0000 7fdef09e5640 1 -- 172.21.15.96:0/2406893039 >> 172.21.15.96:0/2406893039 conn(0x7fdeec098a60 msgr2=0x7fdeec09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- 172.21.15.96:0/2406893039 shutdown_connections 2024-09-13T17:45:21.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- 172.21.15.96:0/2406893039 wait complete. 2024-09-13T17:45:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 Processor -- start 2024-09-13T17:45:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- start start 2024-09-13T17:45:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec13bb90 con 0x7fdeec134e70 2024-09-13T17:45:21.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdeea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdeea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60692/0 (socket says 172.21.15.96:60692) 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdeea59c640 1 -- 172.21.15.96:0/1956183944 learned_addr learned my addr 172.21.15.96:0/1956183944 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdeea59c640 1 -- 172.21.15.96:0/1956183944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee4009420 con 0x7fdeec134e70 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdeea59c640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdee4019030 tx=0x7fdee401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee40abde0 con 0x7fdeec134e70 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4004030 con 0x7fdeec134e70 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdee40042d0 con 0x7fdeec134e70 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdeec0a2e20 con 0x7fdeec134e70 2024-09-13T17:45:21.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec0a1b90 con 0x7fdeec134e70 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fdee4003620 con 0x7fdeec134e70 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec1368b0 con 0x7fdeec134e70 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 0x7fdecc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7fdee40e2590 con 0x7fdeec134e70 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.482+0000 7fdee9d9b640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 0x7fdecc0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:21.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.486+0000 7fdee9d9b640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 0x7fdecc0379a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000f90 tx=0x7fdedc0016a0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:21.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.490+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdee40aa970 con 0x7fdeec134e70 2024-09-13T17:45:21.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.658+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdeec0a2bf0 con 0x7fdeec134e70 2024-09-13T17:45:21.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.658+0000 7fdedaffd640 1 -- 172.21.15.96:0/1956183944 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdee40b0070 con 0x7fdeec134e70 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 msgr2=0x7fdecc0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 0x7fdecc0379a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdedc000f90 tx=0x7fdedc0016a0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 msgr2=0x7fdeec0a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdee4019030 tx=0x7fdee401ac60 comp rx=0 tx=0).stop 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 shutdown_connections 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdecc0354e0 0x7fdecc0379a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 --2- 172.21.15.96:0/1956183944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdeec134e70 0x7fdeec0a25f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 >> 172.21.15.96:0/1956183944 conn(0x7fdeec098a60 msgr2=0x7fdeec131da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 shutdown_connections 2024-09-13T17:45:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:21.662+0000 7fdef09e5640 1 -- 172.21.15.96:0/1956183944 wait complete. 2024-09-13T17:45:21.686 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:45:21.686 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:45:21.776 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:45:21.776 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:21.777 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:21.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:45:21.782 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:21.782 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:21.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:45:21.831 INFO:tasks.cephfs.cephfs_test_case:[, ] 2024-09-13T17:45:21.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:21.941 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:21.934+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:22.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef10a02640 1 Processor -- start 2024-09-13T17:45:22.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef10a02640 1 -- start start 2024-09-13T17:45:22.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef10a02640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fef0c09d690 0x7fef0c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef10a02640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c138050 con 0x7fef0c1387d0 2024-09-13T17:45:22.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef10a02640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c09c410 con 0x7fef0c09d690 2024-09-13T17:45:22.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef0a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fef0c09d690 0x7fef0c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef09d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fef0c1387d0 0x7fef0c134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:32858/0 (socket says 172.21.15.96:32858) 2024-09-13T17:45:22.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef09d9b640 1 -- 172.21.15.96:0/3923120136 learned_addr learned my addr 172.21.15.96:0/3923120136 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:22.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef0a59c640 1 -- 172.21.15.96:0/3923120136 >> v1:172.21.15.96:6789/0 conn(0x7fef0c1387d0 legacy=0x7fef0c134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef0959a640 1 -- 172.21.15.96:0/3923120136 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695388966 0 0) 0x7fef00001120 con 0x7fef0c1387d0 2024-09-13T17:45:22.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.002+0000 7fef0a59c640 1 -- 172.21.15.96:0/3923120136 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c09cba0 con 0x7fef0c09d690 2024-09-13T17:45:22.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 --2- 172.21.15.96:0/3923120136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c138290 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7feefc009740 tx=0x7feefc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40c7b5d4ebef558 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0959a640 1 -- 172.21.15.96:0/3923120136 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feefc0a8840 con 0x7fef0c09d690 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0959a640 1 -- 172.21.15.96:0/3923120136 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0a89a0 con 0x7fef0c09d690 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0959a640 1 -- 172.21.15.96:0/3923120136 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feefc0a8c60 con 0x7fef0c09d690 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/3923120136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 msgr2=0x7fef0c138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 --2- 172.21.15.96:0/3923120136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c138290 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7feefc009740 tx=0x7feefc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/3923120136 shutdown_connections 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 --2- 172.21.15.96:0/3923120136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c138290 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/3923120136 >> 172.21.15.96:0/3923120136 conn(0x7fef0c06bc60 msgr2=0x7fef0c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/3923120136 shutdown_connections 2024-09-13T17:45:22.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/3923120136 wait complete. 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 Processor -- start 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- start start 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0a5dd0 con 0x7fef0c09d690 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:60710/0 (socket says 172.21.15.96:60710) 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 -- 172.21.15.96:0/350905154 learned_addr learned my addr 172.21.15.96:0/350905154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 -- 172.21.15.96:0/350905154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc009420 con 0x7fef0c09d690 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef0a59c640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7feefc01c840 tx=0x7feefc01b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feefc0b0840 con 0x7fef0c09d690 2024-09-13T17:45:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0b09a0 con 0x7fef0c09d690 2024-09-13T17:45:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c0a60e0 con 0x7fef0c09d690 2024-09-13T17:45:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feefc0b0c60 con 0x7fef0c09d690 2024-09-13T17:45:22.015 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.006+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c0a42a0 con 0x7fef0c09d690 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7feefc0b7040 con 0x7fef0c09d690 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c09bc00 con 0x7fef0c09d690 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7feefaffd640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 0x7feeec0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 3404+0+0 (secure 0 0 0) 0x7feefc0e53b0 con 0x7fef0c09d690 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7fef09d9b640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 0x7feeec0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.010+0000 7fef09d9b640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 0x7feeec0379a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef00005890 tx=0x7fef00001280 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.020 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.014+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fef0c09bc00 con 0x7fef0c09d690 2024-09-13T17:45:22.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.218+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef0c09bc00 con 0x7fef0c09d690 2024-09-13T17:45:22.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7feefaffd640 1 -- 172.21.15.96:0/350905154 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2015 (secure 0 0 0) 0x7fef0c09bc00 con 0x7fef0c09d690 2024-09-13T17:45:22.227 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:22.228 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5160,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/1884783954","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":1884783954},{"type":"v1","addr":"172.21.15.96:6827","nonce":1884783954}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30},{"gid":5336,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/3595128332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3595128332},{"type":"v1","addr":"172.21.15.96:6829","nonce":3595128332}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[]} 2024-09-13T17:45:22.228 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 37 2024-09-13T17:45:22.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 msgr2=0x7feeec0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 0x7feeec0379a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fef00005890 tx=0x7fef00001280 comp rx=0 tx=0).stop 2024-09-13T17:45:22.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 msgr2=0x7fef0c0a6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7feefc01c840 tx=0x7feefc01b9b0 comp rx=0 tx=0).stop 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 shutdown_connections 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7feeec0354e0 0x7feeec0379a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 --2- 172.21.15.96:0/350905154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef0c09d690 0x7fef0c0a6b20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 >> 172.21.15.96:0/350905154 conn(0x7fef0c06bc60 msgr2=0x7fef0c1bfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 shutdown_connections 2024-09-13T17:45:22.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.222+0000 7fef10a02640 1 -- 172.21.15.96:0/350905154 wait complete. 2024-09-13T17:45:22.247 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-13T17:45:22.247 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-13T17:45:22.248 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:45:22.248 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:45:22.248 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-13T17:45:22.248 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-13T17:45:22.248 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:45:22.249 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:45:22.258 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:45:22.258 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-13T17:45:22.260 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:45:22.260 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-13T17:45:22.307 INFO:tasks.ceph.mds.a:Started 2024-09-13T17:45:22.308 DEBUG:teuthology.parallel:result is None 2024-09-13T17:45:22.309 INFO:tasks.ceph.mds.b:Started 2024-09-13T17:45:22.309 DEBUG:teuthology.parallel:result is None 2024-09-13T17:45:22.310 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-13T17:45:22.374 INFO:tasks.ceph.mds.a.smithi096.stdout:starting mds.a at 2024-09-13T17:45:22.380 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:45:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.422+0000 7ff963588640 1 Processor -- start 2024-09-13T17:45:22.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.422+0000 7ff963588640 1 -- start start 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff95c13af50 0x7ff95c13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95c138050 con 0x7ff95c09d690 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff95c09c410 con 0x7ff95c13af50 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff960b23640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff95c13af50 0x7ff95c13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff961324640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff95c09d690 0x7ff95c138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52134/0 (socket says 172.21.15.96:52134) 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff960b23640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff95c13af50 0x7ff95c13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45700/0 (socket says 172.21.15.96:45700) 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff961324640 1 -- 172.21.15.96:0/2431148364 learned_addr learned my addr 172.21.15.96:0/2431148364 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff960b23640 1 -- 172.21.15.96:0/2431148364 >> v1:172.21.15.96:6789/0 conn(0x7ff95c09d690 legacy=0x7ff95c138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff960b23640 1 -- 172.21.15.96:0/2431148364 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c09cba0 con 0x7ff95c13af50 2024-09-13T17:45:22.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff953fff640 1 -- 172.21.15.96:0/2431148364 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753240257 0 0) 0x7ff958001120 con 0x7ff95c09d690 2024-09-13T17:45:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff960b23640 1 --2- 172.21.15.96:0/2431148364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c13af50 0x7ff95c13d340 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009740 tx=0x7ff94c01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04a23e7c5d18f9e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff953fff640 1 -- 172.21.15.96:0/2431148364 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff94c0a8840 con 0x7ff95c13af50 2024-09-13T17:45:22.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff953fff640 1 -- 172.21.15.96:0/2431148364 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94c0a89a0 con 0x7ff95c13af50 2024-09-13T17:45:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff953fff640 1 -- 172.21.15.96:0/2431148364 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff94c0a8c60 con 0x7ff95c13af50 2024-09-13T17:45:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- 172.21.15.96:0/2431148364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c13af50 msgr2=0x7ff95c13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:22.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 --2- 172.21.15.96:0/2431148364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c13af50 0x7ff95c13d340 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7ff94c009740 tx=0x7ff94c01ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:22.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- 172.21.15.96:0/2431148364 shutdown_connections 2024-09-13T17:45:22.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 --2- 172.21.15.96:0/2431148364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c13af50 0x7ff95c13d340 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:22.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- 172.21.15.96:0/2431148364 >> 172.21.15.96:0/2431148364 conn(0x7ff95c06bc60 msgr2=0x7ff95c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:22.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- 172.21.15.96:0/2431148364 shutdown_connections 2024-09-13T17:45:22.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.426+0000 7ff963588640 1 -- 172.21.15.96:0/2431148364 wait complete. 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 Processor -- start 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 -- start start 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95c0a6f20 con 0x7ff95c09d690 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff961324640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff961324640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45704/0 (socket says 172.21.15.96:45704) 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff961324640 1 -- 172.21.15.96:0/3853579449 learned_addr learned my addr 172.21.15.96:0/3853579449 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:22.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff961324640 1 -- 172.21.15.96:0/3853579449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c009420 con 0x7ff95c09d690 2024-09-13T17:45:22.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff961324640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c390 tx=0x7ff95800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff958007410 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff958007590 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95c0a4cb0 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9580047f0 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95c0a4ee0 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95c0a4430 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7ff958004950 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 0x7ff9400379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 3404+0+0 (secure 0 0 0) 0x7ff958047310 con 0x7ff95c09d690 2024-09-13T17:45:22.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff960b23640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 0x7ff9400379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:22.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.430+0000 7ff960b23640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 0x7ff9400379a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0b2000 tx=0x7ff94c0aa070 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:22.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.438+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff95c0a4430 con 0x7ff95c09d690 2024-09-13T17:45:22.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:22.618+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff95c138050 con 0x7ff95c09d690 2024-09-13T17:45:22.724 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:22.718+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:23.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.414+0000 7ff951ffb640 1 -- 172.21.15.96:0/3853579449 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff95c138050 con 0x7ff95c09d690 2024-09-13T17:45:23.421 INFO:teuthology.orchestra.run.smithi096.stderr: removed all blocklist entries 2024-09-13T17:45:23.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 msgr2=0x7ff9400379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 0x7ff9400379a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff94c0b2000 tx=0x7ff94c0aa070 comp rx=0 tx=0).stop 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 msgr2=0x7ff95c0a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c390 tx=0x7ff95800c850 comp rx=0 tx=0).stop 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 shutdown_connections 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff9400354e0 0x7ff9400379a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 --2- 172.21.15.96:0/3853579449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff95c09d690 0x7ff95c0a69e0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 >> 172.21.15.96:0/3853579449 conn(0x7ff95c06bc60 msgr2=0x7ff95c139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 shutdown_connections 2024-09-13T17:45:23.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.418+0000 7ff963588640 1 -- 172.21.15.96:0/3853579449 wait complete. 2024-09-13T17:45:23.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-13T17:45:23.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 Processor -- start 2024-09-13T17:45:23.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- start start 2024-09-13T17:45:23.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f363009d690 0x7f3630138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3630138050 con 0x7f36301387d0 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363009c410 con 0x7f363009d690 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362e59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f363009d690 0x7f3630138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362dd9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f36301387d0 0x7f3630134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52142/0 (socket says 172.21.15.96:52142) 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362dd9b640 1 -- 172.21.15.96:0/3961102831 learned_addr learned my addr 172.21.15.96:0/3961102831 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362e59c640 1 -- 172.21.15.96:0/3961102831 >> v1:172.21.15.96:6789/0 conn(0x7f36301387d0 legacy=0x7f3630134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362e59c640 1 -- 172.21.15.96:0/3961102831 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363009cba0 con 0x7f363009d690 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362e59c640 1 --2- 172.21.15.96:0/3961102831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f3630138290 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3628009d90 tx=0x7f362801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d713f0695fcce81a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362d59a640 1 -- 172.21.15.96:0/3961102831 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36280a8e10 con 0x7f363009d690 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362d59a640 1 -- 172.21.15.96:0/3961102831 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280b0440 con 0x7f363009d690 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f362d59a640 1 -- 172.21.15.96:0/3961102831 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36280b0700 con 0x7f363009d690 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- 172.21.15.96:0/3961102831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 msgr2=0x7f3630138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3961102831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f3630138290 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3628009d90 tx=0x7f362801f150 comp rx=0 tx=0).stop 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- 172.21.15.96:0/3961102831 shutdown_connections 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3961102831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f3630138290 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- 172.21.15.96:0/3961102831 >> 172.21.15.96:0/3961102831 conn(0x7f363006bc60 msgr2=0x7f363006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- 172.21.15.96:0/3961102831 shutdown_connections 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.562+0000 7f3634ed3640 1 -- 172.21.15.96:0/3961102831 wait complete. 2024-09-13T17:45:23.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 Processor -- start 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 -- start start 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36300a5d10 con 0x7f363009d690 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45724/0 (socket says 172.21.15.96:45724) 2024-09-13T17:45:23.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362e59c640 1 -- 172.21.15.96:0/3979538063 learned_addr learned my addr 172.21.15.96:0/3979538063 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362e59c640 1 -- 172.21.15.96:0/3979538063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628009a70 con 0x7f363009d690 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362e59c640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f362801cd80 tx=0x7f362801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36280ba540 con 0x7f363009d690 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280ba6a0 con 0x7f363009d690 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36300a4470 con 0x7f363009d690 2024-09-13T17:45:23.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36280ba940 con 0x7f363009d690 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363009f490 con 0x7f363009d690 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363009bc00 con 0x7f363009d690 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f36280b7050 con 0x7f363009d690 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 0x7f36100379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f36280e60e0 con 0x7f363009d690 2024-09-13T17:45:23.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362dd9b640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 0x7f36100379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.566+0000 7f362dd9b640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 0x7f36100379a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3620005890 tx=0x7f3620007a90 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.570+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f363009bc00 con 0x7f363009d690 2024-09-13T17:45:23.588 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:23.582+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:23.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.798+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f363009bc00 con 0x7f363009d690 2024-09-13T17:45:23.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.798+0000 7f361effd640 1 -- 172.21.15.96:0/3979538063 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+2534 (secure 0 0 0) 0x7f363009bc00 con 0x7f363009d690 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "auth_dump": [ 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.a", 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmAVcqLxAAIRo/psndnU3y/VG+/goNRQ==", 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.806 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mds.b", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmpF6ANxAAzrn1jVVSCWKv5q1kdChbWA==", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.0", 2024-09-13T17:45:23.807 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBKeeRmkXcoFhAAdX1HBKKYCHqtacYyAZbF2w==", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.1", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBNeeRmh189GxAAFDpo1k0lxmmillsXoiKXMA==", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:45:23.808 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "osd.2", 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBQeeRmKpvLIhAA4Y7okMAZAQ9cjYpkaKF/kA==", 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow profile osd", 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile osd", 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.809 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.0", 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRm3enLAxAAZGIwl+Bs06GY4JG0dbxLBQ==", 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow r", 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-13T17:45:23.810 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.1", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBIeeRmNlIQCRAAFFcBZ0Pwtq8kXtBAAFkuDA==", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow r", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow rw", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow rwx" 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "client.admin", 2024-09-13T17:45:23.811 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmUwoTGBAAycvCNfOc1ueY+QjBkpAqpA==", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr": "allow *", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow *", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "entity": "mgr.x", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "key": "AQBHeeRmOOcEKhAAJmefIyHpVdx4zDzhwQcawA==", 2024-09-13T17:45:23.812 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": { 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": "allow *", 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "mon": "allow profile mgr", 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: "osd": "allow *" 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-13T17:45:23.813 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 msgr2=0x7f36100379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 0x7f36100379a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3620005890 tx=0x7f3620007a90 comp rx=0 tx=0).stop 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 msgr2=0x7f36300a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f362801cd80 tx=0x7f362801bef0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 shutdown_connections 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f36100354e0 0x7f36100379a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 --2- 172.21.15.96:0/3979538063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f363009d690 0x7f36300a6a60 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 >> 172.21.15.96:0/3979538063 conn(0x7f363006bc60 msgr2=0x7f3630099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 shutdown_connections 2024-09-13T17:45:23.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.802+0000 7f3634ed3640 1 -- 172.21.15.96:0/3979538063 wait complete. 2024-09-13T17:45:23.825 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:23.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.938+0000 7f09442ca640 1 Processor -- start 2024-09-13T17:45:23.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.938+0000 7f09442ca640 1 -- start start 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f093c06f6d0 0x7f093c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f093c06e130 con 0x7f093c09dc20 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f093c0a6750 con 0x7f093c06f6d0 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f093c06f6d0 0x7f093c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f093c06f6d0 0x7f093c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45730/0 (socket says 172.21.15.96:45730) 2024-09-13T17:45:23.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0941865640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f093c09dc20 0x7f093c134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52156/0 (socket says 172.21.15.96:52156) 2024-09-13T17:45:23.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 -- 172.21.15.96:0/2973630784 learned_addr learned my addr 172.21.15.96:0/2973630784 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:23.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 -- 172.21.15.96:0/2973630784 >> v1:172.21.15.96:6789/0 conn(0x7f093c09dc20 legacy=0x7f093c134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.948 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 -- 172.21.15.96:0/2973630784 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c0712e0 con 0x7f093c06f6d0 2024-09-13T17:45:23.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0942066640 1 --2- 172.21.15.96:0/2973630784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c134ab0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f093800ba90 tx=0x7f093801eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9b632b1cad412 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0941064640 1 -- 172.21.15.96:0/2973630784 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09380a77f0 con 0x7f093c06f6d0 2024-09-13T17:45:23.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0941064640 1 -- 172.21.15.96:0/2973630784 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09380a7950 con 0x7f093c06f6d0 2024-09-13T17:45:23.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f0941064640 1 -- 172.21.15.96:0/2973630784 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09380a7c10 con 0x7f093c06f6d0 2024-09-13T17:45:23.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- 172.21.15.96:0/2973630784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 msgr2=0x7f093c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:23.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 --2- 172.21.15.96:0/2973630784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c134ab0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f093800ba90 tx=0x7f093801eb00 comp rx=0 tx=0).stop 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- 172.21.15.96:0/2973630784 shutdown_connections 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 --2- 172.21.15.96:0/2973630784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c134ab0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- 172.21.15.96:0/2973630784 >> 172.21.15.96:0/2973630784 conn(0x7f093c098a60 msgr2=0x7f093c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- 172.21.15.96:0/2973630784 shutdown_connections 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.942+0000 7f09442ca640 1 -- 172.21.15.96:0/2973630784 wait complete. 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 Processor -- start 2024-09-13T17:45:23.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 -- start start 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093c11e3a0 con 0x7f093c06f6d0 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0942066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0942066640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45744/0 (socket says 172.21.15.96:45744) 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0942066640 1 -- 172.21.15.96:0/3183120612 learned_addr learned my addr 172.21.15.96:0/3183120612 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0942066640 1 -- 172.21.15.96:0/3183120612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093800b770 con 0x7f093c06f6d0 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0942066640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f093801c730 tx=0x7f093801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09380be070 con 0x7f093c06f6d0 2024-09-13T17:45:23.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09380afa20 con 0x7f093c06f6d0 2024-09-13T17:45:23.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c120070 con 0x7f093c06f6d0 2024-09-13T17:45:23.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09380afce0 con 0x7f093c06f6d0 2024-09-13T17:45:23.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093c133aa0 con 0x7f093c06f6d0 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f09380c3410 con 0x7f093c06f6d0 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093c1207e0 con 0x7f093c06f6d0 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 0x7f09200379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f09380e2060 con 0x7f093c06f6d0 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.946+0000 7f0941865640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 0x7f09200379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:23.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.950+0000 7f0941865640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 0x7f09200379a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f092c001600 tx=0x7f092c007a50 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:23.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:23.954+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09380ad340 con 0x7f093c06f6d0 2024-09-13T17:45:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.118+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f093c09ae60 con 0x7f093c06f6d0 2024-09-13T17:45:24.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f0932ffd640 1 -- 172.21.15.96:0/3183120612 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+6724 (secure 0 0 0) 0x7f09380b4020 con 0x7f093c06f6d0 2024-09-13T17:45:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:24.127 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":64,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:23.414497+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 msgr2=0x7f09200379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 0x7f09200379a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f092c001600 tx=0x7f092c007a50 comp rx=0 tx=0).stop 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 msgr2=0x7f093c1cfd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f093801c730 tx=0x7f093801d340 comp rx=0 tx=0).stop 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 shutdown_connections 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09200354e0 0x7f09200379a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 --2- 172.21.15.96:0/3183120612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f093c06f6d0 0x7f093c1cfd80 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 >> 172.21.15.96:0/3183120612 conn(0x7f093c098a60 msgr2=0x7f093c132ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 shutdown_connections 2024-09-13T17:45:24.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.122+0000 7f09442ca640 1 -- 172.21.15.96:0/3183120612 wait complete. 2024-09-13T17:45:24.146 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:24.147 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:24.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 Processor -- start 2024-09-13T17:45:24.277 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- start start 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f719809d690 0x7f7198138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7198138050 con 0x7f71981387d0 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719809c410 con 0x7f719809d690 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f719809d690 0x7f7198138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f7195d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f71981387d0 0x7f7198134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52168/0 (socket says 172.21.15.96:52168) 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f7195d9b640 1 -- 172.21.15.96:0/2424073838 learned_addr learned my addr 172.21.15.96:0/2424073838 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:24.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719659c640 1 -- 172.21.15.96:0/2424073838 >> v1:172.21.15.96:6789/0 conn(0x7f71981387d0 legacy=0x7f7198134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719559a640 1 -- 172.21.15.96:0/2424073838 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993936721 0 0) 0x7f718c001120 con 0x7f71981387d0 2024-09-13T17:45:24.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719659c640 1 -- 172.21.15.96:0/2424073838 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719809cba0 con 0x7f719809d690 2024-09-13T17:45:24.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719659c640 1 --2- 172.21.15.96:0/2424073838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f7198138290 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7188009740 tx=0x7f718801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2a4d6129d662f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719559a640 1 -- 172.21.15.96:0/2424073838 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71880a8840 con 0x7f719809d690 2024-09-13T17:45:24.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719559a640 1 -- 172.21.15.96:0/2424073838 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71880a89a0 con 0x7f719809d690 2024-09-13T17:45:24.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719559a640 1 -- 172.21.15.96:0/2424073838 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71880a8c60 con 0x7f719809d690 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- 172.21.15.96:0/2424073838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 msgr2=0x7f7198138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 --2- 172.21.15.96:0/2424073838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f7198138290 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7188009740 tx=0x7f718801ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- 172.21.15.96:0/2424073838 shutdown_connections 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 --2- 172.21.15.96:0/2424073838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f7198138290 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- 172.21.15.96:0/2424073838 >> 172.21.15.96:0/2424073838 conn(0x7f719806bc60 msgr2=0x7f719806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- 172.21.15.96:0/2424073838 shutdown_connections 2024-09-13T17:45:24.282 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.274+0000 7f719cc98640 1 -- 172.21.15.96:0/2424073838 wait complete. 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 Processor -- start 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 -- start start 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71980a6f30 con 0x7f719809d690 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.283 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45758/0 (socket says 172.21.15.96:45758) 2024-09-13T17:45:24.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719659c640 1 -- 172.21.15.96:0/1831607815 learned_addr learned my addr 172.21.15.96:0/1831607815 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:24.284 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719659c640 1 -- 172.21.15.96:0/1831607815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7188009420 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719659c640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f718801b820 tx=0x7f718801d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7188004150 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71880042b0 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71980a5fd0 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71880bc720 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71980a4470 con 0x7f719809d690 2024-09-13T17:45:24.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f71880bc880 con 0x7f719809d690 2024-09-13T17:45:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c005260 con 0x7f719809d690 2024-09-13T17:45:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 0x7f7178037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f71880e53f0 con 0x7f719809d690 2024-09-13T17:45:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7195d9b640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 0x7f7178037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.278+0000 7f7195d9b640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 0x7f7178037950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f718c000c10 tx=0x7f718c001260 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.282+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f71880c4be0 con 0x7f719809d690 2024-09-13T17:45:24.366 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:24.358+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:24.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.454+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f717c005a90 con 0x7f719809d690 2024-09-13T17:45:24.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f7186ffd640 1 -- 172.21.15.96:0/1831607815 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71880c00a0 con 0x7f719809d690 2024-09-13T17:45:24.463 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 msgr2=0x7f7178037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 0x7f7178037950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f718c000c10 tx=0x7f718c001260 comp rx=0 tx=0).stop 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 msgr2=0x7f71980a69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f718801b820 tx=0x7f718801d450 comp rx=0 tx=0).stop 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 shutdown_connections 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f7178035490 0x7f7178037950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 --2- 172.21.15.96:0/1831607815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f719809d690 0x7f71980a69f0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 >> 172.21.15.96:0/1831607815 conn(0x7f719806bc60 msgr2=0x7f71980994a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 shutdown_connections 2024-09-13T17:45:24.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.458+0000 7f719cc98640 1 -- 172.21.15.96:0/1831607815 wait complete. 2024-09-13T17:45:24.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-13T17:45:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f1762a50640 1 Processor -- start 2024-09-13T17:45:24.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f1762a50640 1 -- start start 2024-09-13T17:45:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f1762a50640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f175c134e70 0x7f175c09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f1762a50640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f175c09ed60 con 0x7f175c135250 2024-09-13T17:45:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f1762a50640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175c09ddc0 con 0x7f175c134e70 2024-09-13T17:45:24.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f175bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f175c134e70 0x7f175c09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f17537fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f175c135250 0x7f175c135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52182/0 (socket says 172.21.15.96:52182) 2024-09-13T17:45:24.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f17537fe640 1 -- 172.21.15.96:0/1360288035 learned_addr learned my addr 172.21.15.96:0/1360288035 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:24.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f175bfff640 1 -- 172.21.15.96:0/1360288035 >> v1:172.21.15.96:6789/0 conn(0x7f175c135250 legacy=0x7f175c135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.598+0000 7f175bfff640 1 -- 172.21.15.96:0/1360288035 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175c0a68b0 con 0x7f175c134e70 2024-09-13T17:45:24.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 --2- 172.21.15.96:0/1360288035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c09e820 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f174c009740 tx=0x7f174c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70de0e344176b4b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175b7fe640 1 -- 172.21.15.96:0/1360288035 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f174c0ab530 con 0x7f175c134e70 2024-09-13T17:45:24.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175b7fe640 1 -- 172.21.15.96:0/1360288035 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0a7050 con 0x7f175c134e70 2024-09-13T17:45:24.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175b7fe640 1 -- 172.21.15.96:0/1360288035 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f174c0ab7e0 con 0x7f175c134e70 2024-09-13T17:45:24.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- 172.21.15.96:0/1360288035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 msgr2=0x7f175c09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 --2- 172.21.15.96:0/1360288035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c09e820 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f174c009740 tx=0x7f174c01c420 comp rx=0 tx=0).stop 2024-09-13T17:45:24.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- 172.21.15.96:0/1360288035 shutdown_connections 2024-09-13T17:45:24.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 --2- 172.21.15.96:0/1360288035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c09e820 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- 172.21.15.96:0/1360288035 >> 172.21.15.96:0/1360288035 conn(0x7f175c098a60 msgr2=0x7f175c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- 172.21.15.96:0/1360288035 shutdown_connections 2024-09-13T17:45:24.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- 172.21.15.96:0/1360288035 wait complete. 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 Processor -- start 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- start start 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f1762a50640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175c0a1710 con 0x7f175c134e70 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45776/0 (socket says 172.21.15.96:45776) 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 -- 172.21.15.96:0/924688649 learned_addr learned my addr 172.21.15.96:0/924688649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:24.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 -- 172.21.15.96:0/924688649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174c009420 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.602+0000 7f175bfff640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f174c01a050 tx=0x7f174c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f174c0abea0 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0b4410 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c0a3550 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f174c0b46b0 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175c0a4270 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175c0a3350 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f174c0b2040 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 0x7f173c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f174c0e1650 con 0x7f175c134e70 2024-09-13T17:45:24.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17537fe640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 0x7f173c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.606+0000 7f17537fe640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 0x7f173c0379a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1744001610 tx=0x7f1744007ad0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.610+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f174c0bc3b0 con 0x7f175c134e70 2024-09-13T17:45:24.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f175c1368b0 con 0x7f175c134e70 2024-09-13T17:45:24.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f17597fa640 1 -- 172.21.15.96:0/924688649 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7f174c0b0030 con 0x7f175c134e70 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:24.788 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 msgr2=0x7f173c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 0x7f173c0379a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1744001610 tx=0x7f1744007ad0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 msgr2=0x7f175c12fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f174c01a050 tx=0x7f174c0191c0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 shutdown_connections 2024-09-13T17:45:24.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f173c0354e0 0x7f173c0379a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 --2- 172.21.15.96:0/924688649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175c134e70 0x7f175c12fa60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 >> 172.21.15.96:0/924688649 conn(0x7f175c098a60 msgr2=0x7f175c133fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 shutdown_connections 2024-09-13T17:45:24.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.782+0000 7f1762a50640 1 -- 172.21.15.96:0/924688649 wait complete. 2024-09-13T17:45:24.807 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-13T17:45:24.808 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-13T17:45:24.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f83421f6640 1 Processor -- start 2024-09-13T17:45:24.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f83421f6640 1 -- start start 2024-09-13T17:45:24.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f83421f6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f833c09e890 0x7f833c13ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f83421f6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c09e0d0 con 0x7f833c135e80 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f83421f6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c09ef50 con 0x7f833c09e890 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f833affd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f833c09e890 0x7f833c13ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f833b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f833c135e80 0x7f833c136260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52196/0 (socket says 172.21.15.96:52196) 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f833b7fe640 1 -- 172.21.15.96:0/2910894102 learned_addr learned my addr 172.21.15.96:0/2910894102 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f833affd640 1 -- 172.21.15.96:0/2910894102 >> v1:172.21.15.96:6789/0 conn(0x7f833c135e80 legacy=0x7f833c136260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.930+0000 7f833affd640 1 -- 172.21.15.96:0/2910894102 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c09bf40 con 0x7f833c09e890 2024-09-13T17:45:24.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f833affd640 1 --2- 172.21.15.96:0/2910894102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c13ade0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8324009740 tx=0x7f832401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1b27f2ef4da3ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f833a7fc640 1 -- 172.21.15.96:0/2910894102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83240ab550 con 0x7f833c09e890 2024-09-13T17:45:24.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f833a7fc640 1 -- 172.21.15.96:0/2910894102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83240ab6b0 con 0x7f833c09e890 2024-09-13T17:45:24.940 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f833a7fc640 1 -- 172.21.15.96:0/2910894102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83240ab970 con 0x7f833c09e890 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- 172.21.15.96:0/2910894102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 msgr2=0x7f833c13ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 --2- 172.21.15.96:0/2910894102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c13ade0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f8324009740 tx=0x7f832401c420 comp rx=0 tx=0).stop 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- 172.21.15.96:0/2910894102 shutdown_connections 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 --2- 172.21.15.96:0/2910894102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c13ade0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- 172.21.15.96:0/2910894102 >> 172.21.15.96:0/2910894102 conn(0x7f833c098a60 msgr2=0x7f833c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- 172.21.15.96:0/2910894102 shutdown_connections 2024-09-13T17:45:24.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- 172.21.15.96:0/2910894102 wait complete. 2024-09-13T17:45:24.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 Processor -- start 2024-09-13T17:45:24.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- start start 2024-09-13T17:45:24.942 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.934+0000 7f83421f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c13aaf0 con 0x7f833c09e890 2024-09-13T17:45:24.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45792/0 (socket says 172.21.15.96:45792) 2024-09-13T17:45:24.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833b7fe640 1 -- 172.21.15.96:0/1610758883 learned_addr learned my addr 172.21.15.96:0/1610758883 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:24.944 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833b7fe640 1 -- 172.21.15.96:0/1610758883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8324009420 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833b7fe640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f833400ca90 tx=0x7f833400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8334004ca0 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334004e00 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c13a530 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f833400aba0 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c0a5fa0 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c0a4d20 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f833400ad00 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 0x7f831c037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 2587+0+0 (secure 0 0 0) 0x7f8334047360 con 0x7f833c09e890 2024-09-13T17:45:24.945 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833affd640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 0x7f831c037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:24.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.938+0000 7f833affd640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 0x7f831c037950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83240af000 tx=0x7f8324008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:24.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:24.942+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8334010300 con 0x7f833c09e890 2024-09-13T17:45:25.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.106+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f833c06a1b0 con 0x7f833c09e890 2024-09-13T17:45:25.148 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:25.142+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:25.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f8333fff640 1 -- 172.21.15.96:0/1610758883 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8334018030 con 0x7f833c09e890 2024-09-13T17:45:25.425 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-09-13T17:45:25.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 msgr2=0x7f831c037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:25.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 0x7f831c037950 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f83240af000 tx=0x7f8324008040 comp rx=0 tx=0).stop 2024-09-13T17:45:25.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 msgr2=0x7f833c0a4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:25.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f833400ca90 tx=0x7f833400cf50 comp rx=0 tx=0).stop 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 shutdown_connections 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f831c035490 0x7f831c037950 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 --2- 172.21.15.96:0/1610758883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f833c09e890 0x7f833c0a4ff0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 >> 172.21.15.96:0/1610758883 conn(0x7f833c098a60 msgr2=0x7f833c134e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 shutdown_connections 2024-09-13T17:45:25.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.422+0000 7f83421f6640 1 -- 172.21.15.96:0/1610758883 wait complete. 2024-09-13T17:45:25.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-13T17:45:25.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.582+0000 7faaef0ed640 1 Processor -- start 2024-09-13T17:45:25.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.582+0000 7faaef0ed640 1 -- start start 2024-09-13T17:45:25.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faaef0ed640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7faae809c180 0x7faae813ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:25.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faaef0ed640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faae809d040 con 0x7faae8132a10 2024-09-13T17:45:25.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faaef0ed640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faae8133500 con 0x7faae809c180 2024-09-13T17:45:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7faae809c180 0x7faae813ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7faae809c180 0x7faae813ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45796/0 (socket says 172.21.15.96:45796) 2024-09-13T17:45:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 -- 172.21.15.96:0/2748071742 learned_addr learned my addr 172.21.15.96:0/2748071742 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 -- 172.21.15.96:0/2748071742 >> v1:172.21.15.96:6789/0 conn(0x7faae8132a10 legacy=0x7faae8132df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:45:25.588 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 -- 172.21.15.96:0/2748071742 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae809f080 con 0x7faae809c180 2024-09-13T17:45:25.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae7fff640 1 --2- 172.21.15.96:0/2748071742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae813ae50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7faad00097f0 tx=0x7faad001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdb3904fbcba0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:25.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae77fe640 1 -- 172.21.15.96:0/2748071742 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faad00ab530 con 0x7faae809c180 2024-09-13T17:45:25.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae77fe640 1 -- 172.21.15.96:0/2748071742 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00a7050 con 0x7faae809c180 2024-09-13T17:45:25.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.586+0000 7faae77fe640 1 -- 172.21.15.96:0/2748071742 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faad00ab7e0 con 0x7faae809c180 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/2748071742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 msgr2=0x7faae813ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 --2- 172.21.15.96:0/2748071742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae813ae50 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7faad00097f0 tx=0x7faad001c450 comp rx=0 tx=0).stop 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/2748071742 shutdown_connections 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 --2- 172.21.15.96:0/2748071742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae813ae50 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/2748071742 >> 172.21.15.96:0/2748071742 conn(0x7faae806e020 msgr2=0x7faae8070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/2748071742 shutdown_connections 2024-09-13T17:45:25.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/2748071742 wait complete. 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 Processor -- start 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- start start 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae80a4ed0 con 0x7faae809c180 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaece89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaece89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45812/0 (socket says 172.21.15.96:45812) 2024-09-13T17:45:25.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaece89640 1 -- 172.21.15.96:0/1111893118 learned_addr learned my addr 172.21.15.96:0/1111893118 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaece89640 1 -- 172.21.15.96:0/1111893118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad00094d0 con 0x7faae809c180 2024-09-13T17:45:25.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaece89640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7faad800c7f0 tx=0x7faad800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faad8006c90 con 0x7faae809c180 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad8006e10 con 0x7faae809c180 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faad800a960 con 0x7faae809c180 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae80a43f0 con 0x7faae809c180 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae80a3a40 con 0x7faae809c180 2024-09-13T17:45:25.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faae80a3ef0 con 0x7faae809c180 2024-09-13T17:45:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7faad800aac0 con 0x7faae809c180 2024-09-13T17:45:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.590+0000 7faae57fa640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 0x7faac80379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.594+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 2957+0+0 (secure 0 0 0) 0x7faad8046f40 con 0x7faae809c180 2024-09-13T17:45:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.594+0000 7faae7fff640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 0x7faac80379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:25.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.594+0000 7faae7fff640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 0x7faac80379a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faad00ad000 tx=0x7faad00a4f90 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:25.599 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.598+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7faad8019310 con 0x7faae809c180 2024-09-13T17:45:25.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:25.766+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7faae8073a50 con 0x7faae809c180 2024-09-13T17:45:25.982 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:25.978+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:26.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.438+0000 7faae57fa640 1 -- 172.21.15.96:0/1111893118 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7faad8011b70 con 0x7faae809c180 2024-09-13T17:45:26.441 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-09-13T17:45:26.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.442+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 msgr2=0x7faac80379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.442+0000 7faaef0ed640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 0x7faac80379a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faad00ad000 tx=0x7faad00a4f90 comp rx=0 tx=0).stop 2024-09-13T17:45:26.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.442+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 msgr2=0x7faae80a5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.442+0000 7faaef0ed640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7faad800c7f0 tx=0x7faad800ccb0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 shutdown_connections 2024-09-13T17:45:26.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7faac80354e0 0x7faac80379a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 --2- 172.21.15.96:0/1111893118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae809c180 0x7faae80a5bb0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 >> 172.21.15.96:0/1111893118 conn(0x7faae806e020 msgr2=0x7faae8135120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:26.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 shutdown_connections 2024-09-13T17:45:26.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.446+0000 7faaef0ed640 1 -- 172.21.15.96:0/1111893118 wait complete. 2024-09-13T17:45:26.482 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-13T17:45:26.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.606+0000 7fae31b9f640 1 Processor -- start 2024-09-13T17:45:26.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.606+0000 7fae31b9f640 1 -- start start 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae31b9f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fae2c0a4900 0x7fae2c0a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae31b9f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c070600 con 0x7fae2c0a6510 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae31b9f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c070830 con 0x7fae2c0a4900 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fae2c0a4900 0x7fae2c0a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae23fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fae2c0a6510 0x7fae2c12f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52214/0 (socket says 172.21.15.96:52214) 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae23fff640 1 -- 172.21.15.96:0/2036034872 learned_addr learned my addr 172.21.15.96:0/2036034872 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2b7fe640 1 -- 172.21.15.96:0/2036034872 >> v1:172.21.15.96:6789/0 conn(0x7fae2c0a6510 legacy=0x7fae2c12f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2b7fe640 1 -- 172.21.15.96:0/2036034872 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0a5c70 con 0x7fae2c0a4900 2024-09-13T17:45:26.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2affd640 1 -- 172.21.15.96:0/2036034872 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374319668 0 0) 0x7fae2c070600 con 0x7fae2c0a6510 2024-09-13T17:45:26.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2b7fe640 1 --2- 172.21.15.96:0/2036034872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c0a4ce0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009740 tx=0x7fae1c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca82cb67784226 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:26.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2affd640 1 -- 172.21.15.96:0/2036034872 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1c0ab550 con 0x7fae2c0a4900 2024-09-13T17:45:26.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2affd640 1 -- 172.21.15.96:0/2036034872 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0ab6b0 con 0x7fae2c0a4900 2024-09-13T17:45:26.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.610+0000 7fae2affd640 1 -- 172.21.15.96:0/2036034872 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1c0ab970 con 0x7fae2c0a4900 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2036034872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 msgr2=0x7fae2c0a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2036034872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c0a4ce0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009740 tx=0x7fae1c01c420 comp rx=0 tx=0).stop 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2036034872 shutdown_connections 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2036034872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c0a4ce0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2036034872 >> 172.21.15.96:0/2036034872 conn(0x7fae2c06a150 msgr2=0x7fae2c06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2036034872 shutdown_connections 2024-09-13T17:45:26.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2036034872 wait complete. 2024-09-13T17:45:26.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 Processor -- start 2024-09-13T17:45:26.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- start start 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c09b4b0 con 0x7fae2c0a4900 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae2b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae2b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45834/0 (socket says 172.21.15.96:45834) 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae2b7fe640 1 -- 172.21.15.96:0/2509648135 learned_addr learned my addr 172.21.15.96:0/2509648135 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:26.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae2b7fe640 1 -- 172.21.15.96:0/2509648135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c009420 con 0x7fae2c0a4900 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae2b7fe640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0ad040 tx=0x7fae1c01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1c0abe90 con 0x7fae2c0a4900 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0bf410 con 0x7fae2c0a4900 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fae1c0bf710 con 0x7fae2c0a4900 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c098a60 con 0x7fae2c0a4900 2024-09-13T17:45:26.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c09b770 con 0x7fae2c0a4900 2024-09-13T17:45:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0a4d60 con 0x7fae2c0a4900 2024-09-13T17:45:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.614+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fae1c0bf870 con 0x7fae2c0a4900 2024-09-13T17:45:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.618+0000 7fae28ff9640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 0x7fae0c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.618+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 3339+0+0 (secure 0 0 0) 0x7fae1c0e2500 con 0x7fae2c0a4900 2024-09-13T17:45:26.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.618+0000 7fae23fff640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 0x7fae0c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:26.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.618+0000 7fae23fff640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 0x7fae0c0379a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae14007fd0 tx=0x7fae14007920 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:26.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.622+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fae1c0bfb20 con 0x7fae2c0a4900 2024-09-13T17:45:26.693 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:45:26.832 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.830+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fae2c13ab00 con 0x7fae2c0a4900 2024-09-13T17:45:26.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.834+0000 7fae28ff9640 1 -- 172.21.15.96:0/2509648135 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7fae1c0b2310 con 0x7fae2c0a4900 2024-09-13T17:45:26.840 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 12 and data pool 13 2024-09-13T17:45:26.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 msgr2=0x7fae0c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 0x7fae0c0379a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fae14007fd0 tx=0x7fae14007920 comp rx=0 tx=0).stop 2024-09-13T17:45:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 msgr2=0x7fae2c09bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:26.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fae1c0ad040 tx=0x7fae1c01ac60 comp rx=0 tx=0).stop 2024-09-13T17:45:26.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 shutdown_connections 2024-09-13T17:45:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fae0c0354e0 0x7fae0c0379a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 --2- 172.21.15.96:0/2509648135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae2c0a4900 0x7fae2c09bc00 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 >> 172.21.15.96:0/2509648135 conn(0x7fae2c06a150 msgr2=0x7fae2c134bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 shutdown_connections 2024-09-13T17:45:26.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.850+0000 7fae31b9f640 1 -- 172.21.15.96:0/2509648135 wait complete. 2024-09-13T17:45:26.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:26.903 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:26.898+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:27.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddaf50f640 1 Processor -- start 2024-09-13T17:45:27.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddaf50f640 1 -- start start 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddaf50f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdda81329b0 0x7fdda8132d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddaf50f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda81327b0 con 0x7fdda809dc30 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddaf50f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda809bd90 con 0x7fdda81329b0 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddad2ab640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdda81329b0 0x7fdda8132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddacaaa640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdda809dc30 0x7fdda8135980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52224/0 (socket says 172.21.15.96:52224) 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:26.998+0000 7fddacaaa640 1 -- 172.21.15.96:0/963043623 learned_addr learned my addr 172.21.15.96:0/963043623 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddad2ab640 1 -- 172.21.15.96:0/963043623 >> v1:172.21.15.96:6789/0 conn(0x7fdda809dc30 legacy=0x7fdda8135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddad2ab640 1 -- 172.21.15.96:0/963043623 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda809b2a0 con 0x7fdda81329b0 2024-09-13T17:45:27.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddad2ab640 1 --2- 172.21.15.96:0/963043623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda81329b0 0x7fdda8132d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009d90 tx=0x7fdda001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb10fc03c4214c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fdd9ffff640 1 -- 172.21.15.96:0/963043623 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda00a56b0 con 0x7fdda81329b0 2024-09-13T17:45:27.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fdd9ffff640 1 -- 172.21.15.96:0/963043623 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda00a5810 con 0x7fdda81329b0 2024-09-13T17:45:27.005 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fdd9ffff640 1 -- 172.21.15.96:0/963043623 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda00a5ad0 con 0x7fdda81329b0 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 -- 172.21.15.96:0/963043623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda81329b0 msgr2=0x7fdda8132d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 --2- 172.21.15.96:0/963043623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda81329b0 0x7fdda8132d90 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009d90 tx=0x7fdda001c970 comp rx=0 tx=0).stop 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 -- 172.21.15.96:0/963043623 shutdown_connections 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 --2- 172.21.15.96:0/963043623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda81329b0 0x7fdda8132d90 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 -- 172.21.15.96:0/963043623 >> 172.21.15.96:0/963043623 conn(0x7fdda806ef80 msgr2=0x7fdda80713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:27.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 -- 172.21.15.96:0/963043623 shutdown_connections 2024-09-13T17:45:27.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.002+0000 7fddaf50f640 1 -- 172.21.15.96:0/963043623 wait complete. 2024-09-13T17:45:27.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 Processor -- start 2024-09-13T17:45:27.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 -- start start 2024-09-13T17:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda80a0030 con 0x7fdda809dc30 2024-09-13T17:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddad2ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddad2ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45858/0 (socket says 172.21.15.96:45858) 2024-09-13T17:45:27.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddad2ab640 1 -- 172.21.15.96:0/3770014618 learned_addr learned my addr 172.21.15.96:0/3770014618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:27.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddad2ab640 1 -- 172.21.15.96:0/3770014618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda0009a70 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddad2ab640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fdda0019580 tx=0x7fdda0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda00ad650 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda00ad7b0 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda8139fe0 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdda00adad0 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda813b830 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fdda00b4050 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda80008e0 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 0x7fdd8c0379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fdda00e26c0 con 0x7fdda809dc30 2024-09-13T17:45:27.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddacaaa640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 0x7fdd8c0379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.006+0000 7fddacaaa640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 0x7fdd8c0379f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdda809e040 tx=0x7fdd98007ad0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.014+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdda00c19a0 con 0x7fdda809dc30 2024-09-13T17:45:27.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.186+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdda809d340 con 0x7fdda809dc30 2024-09-13T17:45:27.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.186+0000 7fdd9dffb640 1 -- 172.21.15.96:0/3770014618 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+10507 (secure 0 0 0) 0x7fdda00b2070 con 0x7fdda809dc30 2024-09-13T17:45:27.190 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:27.190 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":67,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:26.832829+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:27.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 msgr2=0x7fdd8c0379f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 0x7fdd8c0379f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdda809e040 tx=0x7fdd98007ad0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 msgr2=0x7fdda81380c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fdda0019580 tx=0x7fdda0019710 comp rx=0 tx=0).stop 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 shutdown_connections 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdd8c035530 0x7fdd8c0379f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 --2- 172.21.15.96:0/3770014618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdda809dc30 0x7fdda81380c0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 >> 172.21.15.96:0/3770014618 conn(0x7fdda806ef80 msgr2=0x7fdda812f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 shutdown_connections 2024-09-13T17:45:27.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.190+0000 7fddaf50f640 1 -- 172.21.15.96:0/3770014618 wait complete. 2024-09-13T17:45:27.210 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:27.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f929d29c640 1 Processor -- start 2024-09-13T17:45:27.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f929d29c640 1 -- start start 2024-09-13T17:45:27.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f929d29c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9298137490 0x7f9298139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f929d29c640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9298071670 con 0x7f9298134670 2024-09-13T17:45:27.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f929d29c640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9298071360 con 0x7f9298137490 2024-09-13T17:45:27.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f9296ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9298134670 0x7f9298134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52230/0 (socket says 172.21.15.96:52230) 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f9296ffd640 1 -- 172.21.15.96:0/2074332003 learned_addr learned my addr 172.21.15.96:0/2074332003 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f928e7fc640 1 --2- 172.21.15.96:0/2074332003 >> v2:172.21.15.96:3300/0 conn(0x7f9298137490 0x7f9298139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f92967fc640 1 -- 172.21.15.96:0/2074332003 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1674446263 0 0) 0x7f9298071670 con 0x7f9298134670 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f92967fc640 1 -- 172.21.15.96:0/2074332003 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9284003530 con 0x7f9298134670 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f928e7fc640 1 -- 172.21.15.96:0/2074332003 >> v1:172.21.15.96:6789/0 conn(0x7f9298134670 legacy=0x7f9298134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.374+0000 7f928e7fc640 1 -- 172.21.15.96:0/2074332003 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92980a7200 con 0x7f9298137490 2024-09-13T17:45:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f928e7fc640 1 --2- 172.21.15.96:0/2074332003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298137490 0x7f9298139880 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f928000a8e0 tx=0x7f928001d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49a15a18d9c0285a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f92967fc640 1 -- 172.21.15.96:0/2074332003 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92800a9550 con 0x7f9298137490 2024-09-13T17:45:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f92967fc640 1 -- 172.21.15.96:0/2074332003 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92800a96b0 con 0x7f9298137490 2024-09-13T17:45:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f92967fc640 1 -- 172.21.15.96:0/2074332003 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f92800a9950 con 0x7f9298137490 2024-09-13T17:45:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- 172.21.15.96:0/2074332003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298137490 msgr2=0x7f9298139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 --2- 172.21.15.96:0/2074332003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298137490 0x7f9298139880 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f928000a8e0 tx=0x7f928001d760 comp rx=0 tx=0).stop 2024-09-13T17:45:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- 172.21.15.96:0/2074332003 shutdown_connections 2024-09-13T17:45:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 --2- 172.21.15.96:0/2074332003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298137490 0x7f9298139880 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- 172.21.15.96:0/2074332003 >> 172.21.15.96:0/2074332003 conn(0x7f9298098a60 msgr2=0x7f929809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- 172.21.15.96:0/2074332003 shutdown_connections 2024-09-13T17:45:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- 172.21.15.96:0/2074332003 wait complete. 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 Processor -- start 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- start start 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f929d29c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f929809e9d0 con 0x7f9298134670 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f9296ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45870/0 (socket says 172.21.15.96:45870) 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f9296ffd640 1 -- 172.21.15.96:0/428353561 learned_addr learned my addr 172.21.15.96:0/428353561 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.378+0000 7f9296ffd640 1 -- 172.21.15.96:0/428353561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928000a5c0 con 0x7f9298134670 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f9296ffd640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f928800c3d0 tx=0x7f928800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9288005ce0 con 0x7f9298134670 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9288005e40 con 0x7f9298134670 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9288007220 con 0x7f9298134670 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92980a3fb0 con 0x7f9298134670 2024-09-13T17:45:27.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92980a4e50 con 0x7f9298134670 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f9288007440 con 0x7f9298134670 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92980a4bf0 con 0x7f9298134670 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 0x7f9278037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f9288046c00 con 0x7f9298134670 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928e7fc640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 0x7f9278037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.382+0000 7f928e7fc640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 0x7f9278037950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92800b0000 tx=0x7f92800a8000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.386+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f92980a4bf0 con 0x7f9298134670 2024-09-13T17:45:27.565 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.562+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92980a3fb0 con 0x7f9298134670 2024-09-13T17:45:27.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f928ffff640 1 -- 172.21.15.96:0/428353561 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+10507 (secure 0 0 0) 0x7f92980a3fb0 con 0x7f9298134670 2024-09-13T17:45:27.568 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:27.568 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":67,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:26.832829+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 msgr2=0x7f9278037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 0x7f9278037950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f92800b0000 tx=0x7f92800a8000 comp rx=0 tx=0).stop 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 msgr2=0x7f929809d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f928800c3d0 tx=0x7f928800c890 comp rx=0 tx=0).stop 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 shutdown_connections 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f9278035490 0x7f9278037950 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 --2- 172.21.15.96:0/428353561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9298134670 0x7f929809d920 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 >> 172.21.15.96:0/428353561 conn(0x7f9298098a60 msgr2=0x7f9298133090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:27.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 shutdown_connections 2024-09-13T17:45:27.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.566+0000 7f929d29c640 1 -- 172.21.15.96:0/428353561 wait complete. 2024-09-13T17:45:27.586 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-13T17:45:27.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23e038a640 1 Processor -- start 2024-09-13T17:45:27.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23e038a640 1 -- start start 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23e038a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23d80a10d0 0x7f23d813d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23e038a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d80a0430 con 0x7f23d8137ed0 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23e038a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d809fa40 con 0x7f23d80a10d0 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23dd925640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23d80a10d0 0x7f23d813d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23dd925640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f23d80a10d0 0x7f23d813d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45882/0 (socket says 172.21.15.96:45882) 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23de126640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f23d8137ed0 0x7f23d81382b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52244/0 (socket says 172.21.15.96:52244) 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23dd925640 1 -- 172.21.15.96:0/15650665 learned_addr learned my addr 172.21.15.96:0/15650665 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23dd925640 1 -- 172.21.15.96:0/15650665 >> v1:172.21.15.96:6789/0 conn(0x7f23d8137ed0 legacy=0x7f23d81382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f23dd925640 1 -- 172.21.15.96:0/15650665 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d806b3d0 con 0x7f23d80a10d0 2024-09-13T17:45:27.719 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:27.714+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23dd925640 1 --2- 172.21.15.96:0/15650665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d813d470 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097f0 tx=0x7f23c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73702e70c632fe52 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23dd124640 1 -- 172.21.15.96:0/15650665 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c80ab530 con 0x7f23d80a10d0 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23dd124640 1 -- 172.21.15.96:0/15650665 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c80a7050 con 0x7f23d80a10d0 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23dd124640 1 -- 172.21.15.96:0/15650665 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23c80ab7e0 con 0x7f23d80a10d0 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- 172.21.15.96:0/15650665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 msgr2=0x7f23d813d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 --2- 172.21.15.96:0/15650665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d813d470 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f23c80097f0 tx=0x7f23c801c450 comp rx=0 tx=0).stop 2024-09-13T17:45:27.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- 172.21.15.96:0/15650665 shutdown_connections 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 --2- 172.21.15.96:0/15650665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d813d470 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- 172.21.15.96:0/15650665 >> 172.21.15.96:0/15650665 conn(0x7f23d806f270 msgr2=0x7f23d8071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- 172.21.15.96:0/15650665 shutdown_connections 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- 172.21.15.96:0/15650665 wait complete. 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 Processor -- start 2024-09-13T17:45:27.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- start start 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23e038a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80a85c0 con 0x7f23d80a10d0 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.718+0000 7f23de126640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23de126640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45898/0 (socket says 172.21.15.96:45898) 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23de126640 1 -- 172.21.15.96:0/2702996629 learned_addr learned my addr 172.21.15.96:0/2702996629 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23de126640 1 -- 172.21.15.96:0/2702996629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80094d0 con 0x7f23d80a10d0 2024-09-13T17:45:27.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23de126640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c410 tx=0x7f23d400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d4007410 con 0x7f23d80a10d0 2024-09-13T17:45:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d4007570 con 0x7f23d80a10d0 2024-09-13T17:45:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d80a2770 con 0x7f23d80a10d0 2024-09-13T17:45:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23d40047f0 con 0x7f23d80a10d0 2024-09-13T17:45:27.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d80a3290 con 0x7f23d80a10d0 2024-09-13T17:45:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d80a2940 con 0x7f23d80a10d0 2024-09-13T17:45:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f23d4004950 con 0x7f23d80a10d0 2024-09-13T17:45:27.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 0x7f23bc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f23d4047160 con 0x7f23d80a10d0 2024-09-13T17:45:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23dd925640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 0x7f23bc0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.722+0000 7f23dd925640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 0x7f23bc0379a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23c80ad000 tx=0x7f23c8008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:27.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.726+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f23d401ac60 con 0x7f23d80a10d0 2024-09-13T17:45:27.943 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:27.942+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f23d806fa30 con 0x7f23d80a10d0 2024-09-13T17:45:28.349 INFO:tasks.ceph.mds.b.smithi096.stderr:2024-09-13T17:45:28.346+0000 7fafbfbe1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:45:28.617 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:28.614+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:28.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.846+0000 7f23ceffd640 1 -- 172.21.15.96:0/2702996629 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f23d401ac60 con 0x7f23d80a10d0 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 msgr2=0x7f23bc0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 0x7f23bc0379a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23c80ad000 tx=0x7f23c8008040 comp rx=0 tx=0).stop 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 msgr2=0x7f23d80a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f23d400c410 tx=0x7f23d400c8d0 comp rx=0 tx=0).stop 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 shutdown_connections 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f23bc0354e0 0x7f23bc0379a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 --2- 172.21.15.96:0/2702996629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23d80a10d0 0x7f23d80a34f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:28.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 >> 172.21.15.96:0/2702996629 conn(0x7f23d806f270 msgr2=0x7f23d8134050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:28.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 shutdown_connections 2024-09-13T17:45:28.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:28.850+0000 7f23e038a640 1 -- 172.21.15.96:0/2702996629 wait complete. 2024-09-13T17:45:28.873 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-13T17:45:28.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-13T17:45:29.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bfb15640 1 Processor -- start 2024-09-13T17:45:29.023 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bfb15640 1 -- start start 2024-09-13T17:45:29.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bfb15640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f84b8134e70 0x7f84b809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bfb15640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b809ed60 con 0x7f84b8135250 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bfb15640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b809ddc0 con 0x7f84b8134e70 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd8b1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f84b8134e70 0x7f84b809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd0b0640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f84b8135250 0x7f84b8135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52252/0 (socket says 172.21.15.96:52252) 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd8b1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f84b8134e70 0x7f84b809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45914/0 (socket says 172.21.15.96:45914) 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd0b0640 1 -- 172.21.15.96:0/2848408708 learned_addr learned my addr 172.21.15.96:0/2848408708 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd8b1640 1 -- 172.21.15.96:0/2848408708 >> v1:172.21.15.96:6789/0 conn(0x7f84b8135250 legacy=0x7f84b8135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd8b1640 1 -- 172.21.15.96:0/2848408708 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b80a68b0 con 0x7f84b8134e70 2024-09-13T17:45:29.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.022+0000 7f84bd8b1640 1 --2- 172.21.15.96:0/2848408708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b809e820 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009d90 tx=0x7f84b001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d284380802e7b2bd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:29.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bc8af640 1 -- 172.21.15.96:0/2848408708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b00a57b0 con 0x7f84b8134e70 2024-09-13T17:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bc8af640 1 -- 172.21.15.96:0/2848408708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b00a5910 con 0x7f84b8134e70 2024-09-13T17:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bc8af640 1 -- 172.21.15.96:0/2848408708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b00a5bd0 con 0x7f84b8134e70 2024-09-13T17:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/2848408708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 msgr2=0x7f84b809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 --2- 172.21.15.96:0/2848408708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b809e820 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009d90 tx=0x7f84b001c970 comp rx=0 tx=0).stop 2024-09-13T17:45:29.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/2848408708 shutdown_connections 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 --2- 172.21.15.96:0/2848408708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b809e820 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/2848408708 >> 172.21.15.96:0/2848408708 conn(0x7f84b8098a60 msgr2=0x7f84b809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/2848408708 shutdown_connections 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/2848408708 wait complete. 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 Processor -- start 2024-09-13T17:45:29.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- start start 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8070990 con 0x7f84b8134e70 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bd8b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bd8b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45924/0 (socket says 172.21.15.96:45924) 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bd8b1640 1 -- 172.21.15.96:0/3283624981 learned_addr learned my addr 172.21.15.96:0/3283624981 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bd8b1640 1 -- 172.21.15.96:0/3283624981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b0009a70 con 0x7f84b8134e70 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bd8b1640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f84b001a5a0 tx=0x7f84b0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b00b9790 con 0x7f84b8134e70 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b00b98f0 con 0x7f84b8134e70 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f84b00b9bd0 con 0x7f84b8134e70 2024-09-13T17:45:29.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b80a45f0 con 0x7f84b8134e70 2024-09-13T17:45:29.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.026+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b80a57a0 con 0x7f84b8134e70 2024-09-13T17:45:29.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f84b00b9d30 con 0x7f84b8134e70 2024-09-13T17:45:29.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b80a4d40 con 0x7f84b8134e70 2024-09-13T17:45:29.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84b67fc640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 0x7f84a40379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:29.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f84b00e2d20 con 0x7f84b8134e70 2024-09-13T17:45:29.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84bd0b0640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 0x7f84a40379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:29.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.030+0000 7f84bd0b0640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 0x7f84a40379a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001610 tx=0x7f84ac007ad0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:29.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.034+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f84b00e7070 con 0x7f84b8134e70 2024-09-13T17:45:29.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.234+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f84b8070ce0 con 0x7f84b8134e70 2024-09-13T17:45:29.513 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:29.510+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:29.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.842+0000 7f84b67fc640 1 -- 172.21.15.96:0/3283624981 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84b00bd020 con 0x7f84b8134e70 2024-09-13T17:45:29.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 msgr2=0x7f84a40379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 0x7f84a40379a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f84ac001610 tx=0x7f84ac007ad0 comp rx=0 tx=0).stop 2024-09-13T17:45:29.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 msgr2=0x7f84b813b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f84b001a5a0 tx=0x7f84b0019710 comp rx=0 tx=0).stop 2024-09-13T17:45:29.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 shutdown_connections 2024-09-13T17:45:29.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f84a40354e0 0x7f84a40379a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:29.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 --2- 172.21.15.96:0/3283624981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84b8134e70 0x7f84b813b3d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:29.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 >> 172.21.15.96:0/3283624981 conn(0x7f84b8098a60 msgr2=0x7f84b80995e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:29.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 shutdown_connections 2024-09-13T17:45:29.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.846+0000 7f84bfb15640 1 -- 172.21.15.96:0/3283624981 wait complete. 2024-09-13T17:45:29.869 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-13T17:45:29.869 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:29.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff72dfe7640 1 Processor -- start 2024-09-13T17:45:29.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff72dfe7640 1 -- start start 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff72dfe7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff728138590 0x7ff72813d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff72dfe7640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72806ca30 con 0x7ff7281381b0 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff72dfe7640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7280a02e0 con 0x7ff728138590 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff71ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7ff728138590 0x7ff72813d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff7277fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7ff7281381b0 0x7ff7280a0a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52264/0 (socket says 172.21.15.96:52264) 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff7277fe640 1 -- 172.21.15.96:0/1757835416 learned_addr learned my addr 172.21.15.96:0/1757835416 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff726ffd640 1 -- 172.21.15.96:0/1757835416 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574744875 0 0) 0x7ff72806ca30 con 0x7ff7281381b0 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff726ffd640 1 -- 172.21.15.96:0/1757835416 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff714003530 con 0x7ff7281381b0 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff71ffff640 1 -- 172.21.15.96:0/1757835416 >> v1:172.21.15.96:6789/0 conn(0x7ff7281381b0 legacy=0x7ff7280a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff71ffff640 1 -- 172.21.15.96:0/1757835416 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7280a05a0 con 0x7ff728138590 2024-09-13T17:45:29.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff71ffff640 1 --2- 172.21.15.96:0/1757835416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff728138590 0x7ff72813d010 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff710009740 tx=0x7ff71001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15bb176b054b976 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:29.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff726ffd640 1 -- 172.21.15.96:0/1757835416 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7100ab530 con 0x7ff728138590 2024-09-13T17:45:29.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff726ffd640 1 -- 172.21.15.96:0/1757835416 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7100a7050 con 0x7ff728138590 2024-09-13T17:45:29.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.994+0000 7ff726ffd640 1 -- 172.21.15.96:0/1757835416 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7100ab7e0 con 0x7ff728138590 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/1757835416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff728138590 msgr2=0x7ff72813d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/1757835416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff728138590 0x7ff72813d010 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7ff710009740 tx=0x7ff71001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/1757835416 shutdown_connections 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/1757835416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff728138590 0x7ff72813d010 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/1757835416 >> 172.21.15.96:0/1757835416 conn(0x7ff72806ef80 msgr2=0x7ff7280713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:29.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/1757835416 shutdown_connections 2024-09-13T17:45:30.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/1757835416 wait complete. 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 Processor -- start 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- start start 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280a53d0 con 0x7ff7281381b0 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff7277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff7277fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45942/0 (socket says 172.21.15.96:45942) 2024-09-13T17:45:30.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff7277fe640 1 -- 172.21.15.96:0/2778942109 learned_addr learned my addr 172.21.15.96:0/2778942109 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:30.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff7277fe640 1 -- 172.21.15.96:0/2778942109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009420 con 0x7ff7281381b0 2024-09-13T17:45:30.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff7277fe640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff71800c3a0 tx=0x7ff71800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff718005c20 con 0x7ff7281381b0 2024-09-13T17:45:30.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718005d80 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7280a8310 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff718007220 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728070f60 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7ff718007380 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:29.998+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff728137860 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.002+0000 7ff724ff9640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 0x7ff7080379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.002+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7ff718046d00 con 0x7ff7281381b0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.002+0000 7ff71ffff640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 0x7ff7080379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.002+0000 7ff71ffff640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 0x7ff7080379a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7100ad000 tx=0x7ff7100a6000 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.006+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7ff718019b40 con 0x7ff7281381b0 2024-09-13T17:45:30.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.214+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7280a0a30 con 0x7ff7281381b0 2024-09-13T17:45:30.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.214+0000 7ff724ff9640 1 -- 172.21.15.96:0/2778942109 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7ff718017020 con 0x7ff7281381b0 2024-09-13T17:45:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:30.217 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:30.218 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 msgr2=0x7ff7080379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 0x7ff7080379a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7100ad000 tx=0x7ff7100a6000 comp rx=0 tx=0).stop 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 msgr2=0x7ff7280a1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7ff71800c3a0 tx=0x7ff71800c860 comp rx=0 tx=0).stop 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 shutdown_connections 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7ff7080354e0 0x7ff7080379a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 --2- 172.21.15.96:0/2778942109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7281381b0 0x7ff7280a1520 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 >> 172.21.15.96:0/2778942109 conn(0x7ff72806ef80 msgr2=0x7ff7281351f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 shutdown_connections 2024-09-13T17:45:30.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.218+0000 7ff72dfe7640 1 -- 172.21.15.96:0/2778942109 wait complete. 2024-09-13T17:45:30.237 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:30.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50dad36640 1 Processor -- start 2024-09-13T17:45:30.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50dad36640 1 -- start start 2024-09-13T17:45:30.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50dad36640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f50d409e820 0x7f50d413ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50dad36640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d40a68b0 con 0x7f50d4134e70 2024-09-13T17:45:30.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50dad36640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d409ddc0 con 0x7f50d409e820 2024-09-13T17:45:30.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f50d409e820 0x7f50d413ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f50d409e820 0x7f50d413ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45954/0 (socket says 172.21.15.96:45954) 2024-09-13T17:45:30.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d3fff640 1 -- 172.21.15.96:0/1543557565 learned_addr learned my addr 172.21.15.96:0/1543557565 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:30.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d8ad2640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f50d4134e70 0x7f50d4135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52278/0 (socket says 172.21.15.96:52278) 2024-09-13T17:45:30.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d3fff640 1 -- 172.21.15.96:0/1543557565 >> v1:172.21.15.96:6789/0 conn(0x7f50d4134e70 legacy=0x7f50d4135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d3fff640 1 -- 172.21.15.96:0/1543557565 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d4136970 con 0x7f50d409e820 2024-09-13T17:45:30.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.358+0000 7f50d37fe640 1 -- 172.21.15.96:0/1543557565 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989689924 0 0) 0x7f50d40a68b0 con 0x7f50d4134e70 2024-09-13T17:45:30.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d3fff640 1 --2- 172.21.15.96:0/1543557565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d413ae50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097f0 tx=0x7f50c401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9bd4c1f6ed23e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d37fe640 1 -- 172.21.15.96:0/1543557565 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50c40ac540 con 0x7f50d409e820 2024-09-13T17:45:30.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d37fe640 1 -- 172.21.15.96:0/1543557565 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c40ac6a0 con 0x7f50d409e820 2024-09-13T17:45:30.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d37fe640 1 -- 172.21.15.96:0/1543557565 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50c40ac960 con 0x7f50d409e820 2024-09-13T17:45:30.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- 172.21.15.96:0/1543557565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 msgr2=0x7f50d413ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 --2- 172.21.15.96:0/1543557565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d413ae50 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f50c40097f0 tx=0x7f50c401c4e0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- 172.21.15.96:0/1543557565 shutdown_connections 2024-09-13T17:45:30.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 --2- 172.21.15.96:0/1543557565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d413ae50 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- 172.21.15.96:0/1543557565 >> 172.21.15.96:0/1543557565 conn(0x7f50d4098a60 msgr2=0x7f50d409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- 172.21.15.96:0/1543557565 shutdown_connections 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- 172.21.15.96:0/1543557565 wait complete. 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 Processor -- start 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- start start 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50dad36640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d4071570 con 0x7f50d409e820 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d8ad2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d8ad2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45964/0 (socket says 172.21.15.96:45964) 2024-09-13T17:45:30.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.362+0000 7f50d8ad2640 1 -- 172.21.15.96:0/4116183706 learned_addr learned my addr 172.21.15.96:0/4116183706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:30.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d8ad2640 1 -- 172.21.15.96:0/4116183706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40094d0 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d8ad2640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f50c800c420 tx=0x7f50c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50c80074d0 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c8007630 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d4071310 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50c8004960 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d40a42e0 con 0x7f50d409e820 2024-09-13T17:45:30.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f50c8004ac0 con 0x7f50d409e820 2024-09-13T17:45:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 0x7f50c00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f50c8046cb0 con 0x7f50d409e820 2024-09-13T17:45:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d4132f50 con 0x7f50d409e820 2024-09-13T17:45:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d3fff640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 0x7f50c00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.366+0000 7f50d3fff640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 0x7f50c00379a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ae000 tx=0x7f50c4008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.370+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f50c8018070 con 0x7f50d409e820 2024-09-13T17:45:30.384 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:30.382+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:30.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.546+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d406a1b0 con 0x7f50d409e820 2024-09-13T17:45:30.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.546+0000 7f50d17fa640 1 -- 172.21.15.96:0/4116183706 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+10494 (secure 0 0 0) 0x7f50c801ab40 con 0x7f50d409e820 2024-09-13T17:45:30.550 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:30.550 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":68,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:27.834546+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 msgr2=0x7f50c00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 0x7f50c00379a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f50c40ae000 tx=0x7f50c4008040 comp rx=0 tx=0).stop 2024-09-13T17:45:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 msgr2=0x7f50d41cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f50c800c420 tx=0x7f50c800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 shutdown_connections 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f50c00354e0 0x7f50c00379a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 --2- 172.21.15.96:0/4116183706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50d409e820 0x7f50d41cfa20 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 >> 172.21.15.96:0/4116183706 conn(0x7f50d4098a60 msgr2=0x7f50d406dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 shutdown_connections 2024-09-13T17:45:30.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.550+0000 7f50dad36640 1 -- 172.21.15.96:0/4116183706 wait complete. 2024-09-13T17:45:30.570 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-09-13T17:45:30.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:30.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.682+0000 7f8ef37cf640 1 Processor -- start 2024-09-13T17:45:30.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.682+0000 7f8ef37cf640 1 -- start start 2024-09-13T17:45:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8eec134e70 0x7f8eec09e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec09ed60 con 0x7f8eec135250 2024-09-13T17:45:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eec09ddc0 con 0x7f8eec134e70 2024-09-13T17:45:30.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef156b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8eec134e70 0x7f8eec09e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef0d6a640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8eec135250 0x7f8eec135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52288/0 (socket says 172.21.15.96:52288) 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef0d6a640 1 -- 172.21.15.96:0/1576772541 learned_addr learned my addr 172.21.15.96:0/1576772541 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ee3fff640 1 -- 172.21.15.96:0/1576772541 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120668343 0 0) 0x7f8edc001120 con 0x7f8eec135250 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ee3fff640 1 -- 172.21.15.96:0/1576772541 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed8003530 con 0x7f8eec135250 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef156b640 1 -- 172.21.15.96:0/1576772541 >> v1:172.21.15.96:6789/0 conn(0x7f8eec135250 legacy=0x7f8eec135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef156b640 1 -- 172.21.15.96:0/1576772541 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec0a68b0 con 0x7f8eec134e70 2024-09-13T17:45:30.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef156b640 1 --2- 172.21.15.96:0/1576772541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec09e820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009740 tx=0x7f8ee801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09ac4455f2f60ff server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ee3fff640 1 -- 172.21.15.96:0/1576772541 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ee80ab530 con 0x7f8eec134e70 2024-09-13T17:45:30.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ee3fff640 1 -- 172.21.15.96:0/1576772541 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee80a7050 con 0x7f8eec134e70 2024-09-13T17:45:30.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ee3fff640 1 -- 172.21.15.96:0/1576772541 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ee80ab7e0 con 0x7f8eec134e70 2024-09-13T17:45:30.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- 172.21.15.96:0/1576772541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 msgr2=0x7f8eec09e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/1576772541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec09e820 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8009740 tx=0x7f8ee801c3e0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- 172.21.15.96:0/1576772541 shutdown_connections 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/1576772541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec09e820 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- 172.21.15.96:0/1576772541 >> 172.21.15.96:0/1576772541 conn(0x7f8eec098a60 msgr2=0x7f8eec09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- 172.21.15.96:0/1576772541 shutdown_connections 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.686+0000 7f8ef37cf640 1 -- 172.21.15.96:0/1576772541 wait complete. 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 Processor -- start 2024-09-13T17:45:30.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 -- start start 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec132f50 con 0x7f8eec134e70 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef156b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef156b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45986/0 (socket says 172.21.15.96:45986) 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef156b640 1 -- 172.21.15.96:0/2216418177 learned_addr learned my addr 172.21.15.96:0/2216418177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef156b640 1 -- 172.21.15.96:0/2216418177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8009420 con 0x7f8eec134e70 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef156b640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80ae040 tx=0x7f8ee801ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ee80abd70 con 0x7f8eec134e70 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee80b7450 con 0x7f8eec134e70 2024-09-13T17:45:30.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8ee80b75c0 con 0x7f8eec134e70 2024-09-13T17:45:30.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec0a1ff0 con 0x7f8eec134e70 2024-09-13T17:45:30.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec13ba70 con 0x7f8eec134e70 2024-09-13T17:45:30.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eec1368b0 con 0x7f8eec134e70 2024-09-13T17:45:30.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f8ee80b7720 con 0x7f8eec134e70 2024-09-13T17:45:30.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 0x7f8ed00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:30.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8ee80e1f90 con 0x7f8eec134e70 2024-09-13T17:45:30.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.690+0000 7f8ef0d6a640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 0x7f8ed00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:30.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.694+0000 7f8ef0d6a640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 0x7f8ed00379a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001380 tx=0x7f8edc007820 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:30.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.698+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8ee80c76c0 con 0x7f8eec134e70 2024-09-13T17:45:30.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.862+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 --> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8eec136580 con 0x7f8ed00354e0 2024-09-13T17:45:30.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.870+0000 7f8ee1ffb640 1 -- 172.21.15.96:0/2216418177 <== mgr.5260 v2:172.21.15.96:6824/3517445048 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250147 (secure 0 0 0) 0x7f8edc001c50 con 0x7f8ed00354e0 2024-09-13T17:45:30.874 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:30.876 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 msgr2=0x7f8ed00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 0x7f8ed00379a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8edc001380 tx=0x7f8edc007820 comp rx=0 tx=0).stop 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 msgr2=0x7f8eec13aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80ae040 tx=0x7f8ee801ac20 comp rx=0 tx=0).stop 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 shutdown_connections 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8ed00354e0 0x7f8ed00379a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 --2- 172.21.15.96:0/2216418177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8eec134e70 0x7f8eec13aff0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 >> 172.21.15.96:0/2216418177 conn(0x7f8eec098a60 msgr2=0x7f8eec098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 shutdown_connections 2024-09-13T17:45:30.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:30.874+0000 7f8ef37cf640 1 -- 172.21.15.96:0/2216418177 wait complete. 2024-09-13T17:45:30.892 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T17:45:29.545923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":267,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1963,"ondisk_log_size":1963,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":108744,"kb_used_data":3088,"kb_used_omap":4,"kb_used_meta":105595,"kb_avail":283006776,"statfs":{"total":289910292480,"available":289798938624,"internally_reserved":0,"allocated":3162112,"data_stored":1944130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":108129619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.126420"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865488+0000","last_change":"2024-09-13T17:45:26.857597+0000","last_active":"2024-09-13T17:45:26.865488+0000","last_peered":"2024-09-13T17:45:26.865488+0000","last_clean":"2024-09-13T17:45:26.865488+0000","last_became_active":"2024-09-13T17:45:26.857268+0000","last_became_peered":"2024-09-13T17:45:26.857268+0000","last_unstale":"2024-09-13T17:45:26.865488+0000","last_undegraded":"2024-09-13T17:45:26.865488+0000","last_fullsized":"2024-09-13T17:45:26.865488+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:27.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859531+0000","last_change":"2024-09-13T17:45:26.854681+0000","last_active":"2024-09-13T17:45:26.859531+0000","last_peered":"2024-09-13T17:45:26.859531+0000","last_clean":"2024-09-13T17:45:26.859531+0000","last_became_active":"2024-09-13T17:45:26.854327+0000","last_became_peered":"2024-09-13T17:45:26.854327+0000","last_unstale":"2024-09-13T17:45:26.859531+0000","last_undegraded":"2024-09-13T17:45:26.859531+0000","last_fullsized":"2024-09-13T17:45:26.859531+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:02.521318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865388+0000","last_change":"2024-09-13T17:45:26.860765+0000","last_active":"2024-09-13T17:45:26.865388+0000","last_peered":"2024-09-13T17:45:26.865388+0000","last_clean":"2024-09-13T17:45:26.865388+0000","last_became_active":"2024-09-13T17:45:26.860494+0000","last_became_peered":"2024-09-13T17:45:26.860494+0000","last_unstale":"2024-09-13T17:45:26.865388+0000","last_undegraded":"2024-09-13T17:45:26.865388+0000","last_fullsized":"2024-09-13T17:45:26.865388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:48.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865401+0000","last_change":"2024-09-13T17:45:26.860895+0000","last_active":"2024-09-13T17:45:26.865401+0000","last_peered":"2024-09-13T17:45:26.865401+0000","last_clean":"2024-09-13T17:45:26.865401+0000","last_became_active":"2024-09-13T17:45:26.860511+0000","last_became_peered":"2024-09-13T17:45:26.860511+0000","last_unstale":"2024-09-13T17:45:26.865401+0000","last_undegraded":"2024-09-13T17:45:26.865401+0000","last_fullsized":"2024-09-13T17:45:26.865401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:05.910353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863437+0000","last_change":"2024-09-13T17:45:26.858688+0000","last_active":"2024-09-13T17:45:26.863437+0000","last_peered":"2024-09-13T17:45:26.863437+0000","last_clean":"2024-09-13T17:45:26.863437+0000","last_became_active":"2024-09-13T17:45:26.858384+0000","last_became_peered":"2024-09-13T17:45:26.858384+0000","last_unstale":"2024-09-13T17:45:26.863437+0000","last_undegraded":"2024-09-13T17:45:26.863437+0000","last_fullsized":"2024-09-13T17:45:26.863437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:53.025921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881207+0000","last_change":"2024-09-13T17:45:26.875078+0000","last_active":"2024-09-13T17:45:26.881207+0000","last_peered":"2024-09-13T17:45:26.881207+0000","last_clean":"2024-09-13T17:45:26.881207+0000","last_became_active":"2024-09-13T17:45:26.874666+0000","last_became_peered":"2024-09-13T17:45:26.874666+0000","last_unstale":"2024-09-13T17:45:26.881207+0000","last_undegraded":"2024-09-13T17:45:26.881207+0000","last_fullsized":"2024-09-13T17:45:26.881207+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:16.908325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876147+0000","last_change":"2024-09-13T17:45:26.874893+0000","last_active":"2024-09-13T17:45:26.876147+0000","last_peered":"2024-09-13T17:45:26.876147+0000","last_clean":"2024-09-13T17:45:26.876147+0000","last_became_active":"2024-09-13T17:45:26.874643+0000","last_became_peered":"2024-09-13T17:45:26.874643+0000","last_unstale":"2024-09-13T17:45:26.876147+0000","last_undegraded":"2024-09-13T17:45:26.876147+0000","last_fullsized":"2024-09-13T17:45:26.876147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:36.836974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876673+0000","last_change":"2024-09-13T17:45:26.868970+0000","last_active":"2024-09-13T17:45:26.876673+0000","last_peered":"2024-09-13T17:45:26.876673+0000","last_clean":"2024-09-13T17:45:26.876673+0000","last_became_active":"2024-09-13T17:45:26.868681+0000","last_became_peered":"2024-09-13T17:45:26.868681+0000","last_unstale":"2024-09-13T17:45:26.876673+0000","last_undegraded":"2024-09-13T17:45:26.876673+0000","last_fullsized":"2024-09-13T17:45:26.876673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:14.740110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881391+0000","last_change":"2024-09-13T17:45:26.873882+0000","last_active":"2024-09-13T17:45:26.881391+0000","last_peered":"2024-09-13T17:45:26.881391+0000","last_clean":"2024-09-13T17:45:26.881391+0000","last_became_active":"2024-09-13T17:45:26.873611+0000","last_became_peered":"2024-09-13T17:45:26.873611+0000","last_unstale":"2024-09-13T17:45:26.881391+0000","last_undegraded":"2024-09-13T17:45:26.881391+0000","last_fullsized":"2024-09-13T17:45:26.881391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:17.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884059+0000","last_change":"2024-09-13T17:45:26.876739+0000","last_active":"2024-09-13T17:45:26.884059+0000","last_peered":"2024-09-13T17:45:26.884059+0000","last_clean":"2024-09-13T17:45:26.884059+0000","last_became_active":"2024-09-13T17:45:26.876433+0000","last_became_peered":"2024-09-13T17:45:26.876433+0000","last_unstale":"2024-09-13T17:45:26.884059+0000","last_undegraded":"2024-09-13T17:45:26.884059+0000","last_fullsized":"2024-09-13T17:45:26.884059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:42.690789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859632+0000","last_change":"2024-09-13T17:45:26.855741+0000","last_active":"2024-09-13T17:45:26.859632+0000","last_peered":"2024-09-13T17:45:26.859632+0000","last_clean":"2024-09-13T17:45:26.859632+0000","last_became_active":"2024-09-13T17:45:26.855060+0000","last_became_peered":"2024-09-13T17:45:26.855060+0000","last_unstale":"2024-09-13T17:45:26.859632+0000","last_undegraded":"2024-09-13T17:45:26.859632+0000","last_fullsized":"2024-09-13T17:45:26.859632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:28.806397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877242+0000","last_change":"2024-09-13T17:45:26.866296+0000","last_active":"2024-09-13T17:45:26.877242+0000","last_peered":"2024-09-13T17:45:26.877242+0000","last_clean":"2024-09-13T17:45:26.877242+0000","last_became_active":"2024-09-13T17:45:26.863281+0000","last_became_peered":"2024-09-13T17:45:26.863281+0000","last_unstale":"2024-09-13T17:45:26.877242+0000","last_undegraded":"2024-09-13T17:45:26.877242+0000","last_fullsized":"2024-09-13T17:45:26.877242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:58.053260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876190+0000","last_change":"2024-09-13T17:45:26.866601+0000","last_active":"2024-09-13T17:45:26.876190+0000","last_peered":"2024-09-13T17:45:26.876190+0000","last_clean":"2024-09-13T17:45:26.876190+0000","last_became_active":"2024-09-13T17:45:26.866106+0000","last_became_peered":"2024-09-13T17:45:26.866106+0000","last_unstale":"2024-09-13T17:45:26.876190+0000","last_undegraded":"2024-09-13T17:45:26.876190+0000","last_fullsized":"2024-09-13T17:45:26.876190+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:17.165403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863813+0000","last_change":"2024-09-13T17:45:26.858852+0000","last_active":"2024-09-13T17:45:26.863813+0000","last_peered":"2024-09-13T17:45:26.863813+0000","last_clean":"2024-09-13T17:45:26.863813+0000","last_became_active":"2024-09-13T17:45:26.858594+0000","last_became_peered":"2024-09-13T17:45:26.858594+0000","last_unstale":"2024-09-13T17:45:26.863813+0000","last_undegraded":"2024-09-13T17:45:26.863813+0000","last_fullsized":"2024-09-13T17:45:26.863813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:59.760400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.868170+0000","last_change":"2024-09-13T17:45:26.867404+0000","last_active":"2024-09-13T17:45:26.868170+0000","last_peered":"2024-09-13T17:45:26.868170+0000","last_clean":"2024-09-13T17:45:26.868170+0000","last_became_active":"2024-09-13T17:45:26.867102+0000","last_became_peered":"2024-09-13T17:45:26.867102+0000","last_unstale":"2024-09-13T17:45:26.868170+0000","last_undegraded":"2024-09-13T17:45:26.868170+0000","last_fullsized":"2024-09-13T17:45:26.868170+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:20.032657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872631+0000","last_change":"2024-09-13T17:45:26.868827+0000","last_active":"2024-09-13T17:45:26.872631+0000","last_peered":"2024-09-13T17:45:26.872631+0000","last_clean":"2024-09-13T17:45:26.872631+0000","last_became_active":"2024-09-13T17:45:26.868576+0000","last_became_peered":"2024-09-13T17:45:26.868576+0000","last_unstale":"2024-09-13T17:45:26.872631+0000","last_undegraded":"2024-09-13T17:45:26.872631+0000","last_fullsized":"2024-09-13T17:45:26.872631+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:28.806397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875732+0000","last_change":"2024-09-13T17:45:26.869411+0000","last_active":"2024-09-13T17:45:26.875732+0000","last_peered":"2024-09-13T17:45:26.875732+0000","last_clean":"2024-09-13T17:45:26.875732+0000","last_became_active":"2024-09-13T17:45:26.869158+0000","last_became_peered":"2024-09-13T17:45:26.869158+0000","last_unstale":"2024-09-13T17:45:26.875732+0000","last_undegraded":"2024-09-13T17:45:26.875732+0000","last_fullsized":"2024-09-13T17:45:26.875732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:33.306502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884213+0000","last_change":"2024-09-13T17:45:26.877716+0000","last_active":"2024-09-13T17:45:26.884213+0000","last_peered":"2024-09-13T17:45:26.884213+0000","last_clean":"2024-09-13T17:45:26.884213+0000","last_became_active":"2024-09-13T17:45:26.876640+0000","last_became_peered":"2024-09-13T17:45:26.876640+0000","last_unstale":"2024-09-13T17:45:26.884213+0000","last_undegraded":"2024-09-13T17:45:26.884213+0000","last_fullsized":"2024-09-13T17:45:26.884213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:02.521318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875798+0000","last_change":"2024-09-13T17:45:26.871016+0000","last_active":"2024-09-13T17:45:26.875798+0000","last_peered":"2024-09-13T17:45:26.875798+0000","last_clean":"2024-09-13T17:45:26.875798+0000","last_became_active":"2024-09-13T17:45:26.864520+0000","last_became_peered":"2024-09-13T17:45:26.864520+0000","last_unstale":"2024-09-13T17:45:26.875798+0000","last_undegraded":"2024-09-13T17:45:26.875798+0000","last_fullsized":"2024-09-13T17:45:26.875798+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:54:11.472571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884041+0000","last_change":"2024-09-13T17:45:26.876378+0000","last_active":"2024-09-13T17:45:26.884041+0000","last_peered":"2024-09-13T17:45:26.884041+0000","last_clean":"2024-09-13T17:45:26.884041+0000","last_became_active":"2024-09-13T17:45:26.875700+0000","last_became_peered":"2024-09-13T17:45:26.875700+0000","last_unstale":"2024-09-13T17:45:26.884041+0000","last_undegraded":"2024-09-13T17:45:26.884041+0000","last_fullsized":"2024-09-13T17:45:26.884041+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:58.649338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881324+0000","last_change":"2024-09-13T17:45:26.875452+0000","last_active":"2024-09-13T17:45:26.881324+0000","last_peered":"2024-09-13T17:45:26.881324+0000","last_clean":"2024-09-13T17:45:26.881324+0000","last_became_active":"2024-09-13T17:45:26.875203+0000","last_became_peered":"2024-09-13T17:45:26.875203+0000","last_unstale":"2024-09-13T17:45:26.881324+0000","last_undegraded":"2024-09-13T17:45:26.881324+0000","last_fullsized":"2024-09-13T17:45:26.881324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:38.510250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.867552+0000","last_change":"2024-09-13T17:45:26.864287+0000","last_active":"2024-09-13T17:45:26.867552+0000","last_peered":"2024-09-13T17:45:26.867552+0000","last_clean":"2024-09-13T17:45:26.867552+0000","last_became_active":"2024-09-13T17:45:26.863894+0000","last_became_peered":"2024-09-13T17:45:26.863894+0000","last_unstale":"2024-09-13T17:45:26.867552+0000","last_undegraded":"2024-09-13T17:45:26.867552+0000","last_fullsized":"2024-09-13T17:45:26.867552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875535+0000","last_change":"2024-09-13T17:45:26.868513+0000","last_active":"2024-09-13T17:45:26.875535+0000","last_peered":"2024-09-13T17:45:26.875535+0000","last_clean":"2024-09-13T17:45:26.875535+0000","last_became_active":"2024-09-13T17:45:26.864992+0000","last_became_peered":"2024-09-13T17:45:26.864992+0000","last_unstale":"2024-09-13T17:45:26.875535+0000","last_undegraded":"2024-09-13T17:45:26.875535+0000","last_fullsized":"2024-09-13T17:45:26.875535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:33.269069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876119+0000","last_change":"2024-09-13T17:45:26.874888+0000","last_active":"2024-09-13T17:45:26.876119+0000","last_peered":"2024-09-13T17:45:26.876119+0000","last_clean":"2024-09-13T17:45:26.876119+0000","last_became_active":"2024-09-13T17:45:26.874655+0000","last_became_peered":"2024-09-13T17:45:26.874655+0000","last_unstale":"2024-09-13T17:45:26.876119+0000","last_undegraded":"2024-09-13T17:45:26.876119+0000","last_fullsized":"2024-09-13T17:45:26.876119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:50.343310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881348+0000","last_change":"2024-09-13T17:45:26.873644+0000","last_active":"2024-09-13T17:45:26.881348+0000","last_peered":"2024-09-13T17:45:26.881348+0000","last_clean":"2024-09-13T17:45:26.881348+0000","last_became_active":"2024-09-13T17:45:26.873275+0000","last_became_peered":"2024-09-13T17:45:26.873275+0000","last_unstale":"2024-09-13T17:45:26.881348+0000","last_undegraded":"2024-09-13T17:45:26.881348+0000","last_fullsized":"2024-09-13T17:45:26.881348+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:05.368912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881329+0000","last_change":"2024-09-13T17:45:26.873635+0000","last_active":"2024-09-13T17:45:26.881329+0000","last_peered":"2024-09-13T17:45:26.881329+0000","last_clean":"2024-09-13T17:45:26.881329+0000","last_became_active":"2024-09-13T17:45:26.873374+0000","last_became_peered":"2024-09-13T17:45:26.873374+0000","last_unstale":"2024-09-13T17:45:26.881329+0000","last_undegraded":"2024-09-13T17:45:26.881329+0000","last_fullsized":"2024-09-13T17:45:26.881329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:02.504758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884178+0000","last_change":"2024-09-13T17:45:26.877495+0000","last_active":"2024-09-13T17:45:26.884178+0000","last_peered":"2024-09-13T17:45:26.884178+0000","last_clean":"2024-09-13T17:45:26.884178+0000","last_became_active":"2024-09-13T17:45:26.877033+0000","last_became_peered":"2024-09-13T17:45:26.877033+0000","last_unstale":"2024-09-13T17:45:26.884178+0000","last_undegraded":"2024-09-13T17:45:26.884178+0000","last_fullsized":"2024-09-13T17:45:26.884178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:27.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859558+0000","last_change":"2024-09-13T17:45:26.855122+0000","last_active":"2024-09-13T17:45:26.859558+0000","last_peered":"2024-09-13T17:45:26.859558+0000","last_clean":"2024-09-13T17:45:26.859558+0000","last_became_active":"2024-09-13T17:45:26.854850+0000","last_became_peered":"2024-09-13T17:45:26.854850+0000","last_unstale":"2024-09-13T17:45:26.859558+0000","last_undegraded":"2024-09-13T17:45:26.859558+0000","last_fullsized":"2024-09-13T17:45:26.859558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:07:11.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.870908+0000","last_change":"2024-09-13T17:45:26.867972+0000","last_active":"2024-09-13T17:45:26.870908+0000","last_peered":"2024-09-13T17:45:26.870908+0000","last_clean":"2024-09-13T17:45:26.870908+0000","last_became_active":"2024-09-13T17:45:26.867297+0000","last_became_peered":"2024-09-13T17:45:26.867297+0000","last_unstale":"2024-09-13T17:45:26.870908+0000","last_undegraded":"2024-09-13T17:45:26.870908+0000","last_fullsized":"2024-09-13T17:45:26.870908+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:13.541689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859198+0000","last_change":"2024-09-13T17:45:26.850945+0000","last_active":"2024-09-13T17:45:26.859198+0000","last_peered":"2024-09-13T17:45:26.859198+0000","last_clean":"2024-09-13T17:45:26.859198+0000","last_became_active":"2024-09-13T17:45:26.850261+0000","last_became_peered":"2024-09-13T17:45:26.850261+0000","last_unstale":"2024-09-13T17:45:26.859198+0000","last_undegraded":"2024-09-13T17:45:26.859198+0000","last_fullsized":"2024-09-13T17:45:26.859198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:21.495952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881215+0000","last_change":"2024-09-13T17:45:26.875222+0000","last_active":"2024-09-13T17:45:26.881215+0000","last_peered":"2024-09-13T17:45:26.881215+0000","last_clean":"2024-09-13T17:45:26.881215+0000","last_became_active":"2024-09-13T17:45:26.874949+0000","last_became_peered":"2024-09-13T17:45:26.874949+0000","last_unstale":"2024-09-13T17:45:26.881215+0000","last_undegraded":"2024-09-13T17:45:26.881215+0000","last_fullsized":"2024-09-13T17:45:26.881215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:20.032657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865438+0000","last_change":"2024-09-13T17:45:26.861148+0000","last_active":"2024-09-13T17:45:26.865438+0000","last_peered":"2024-09-13T17:45:26.865438+0000","last_clean":"2024-09-13T17:45:26.865438+0000","last_became_active":"2024-09-13T17:45:26.860689+0000","last_became_peered":"2024-09-13T17:45:26.860689+0000","last_unstale":"2024-09-13T17:45:26.865438+0000","last_undegraded":"2024-09-13T17:45:26.865438+0000","last_fullsized":"2024-09-13T17:45:26.865438+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:29.009219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875556+0000","last_change":"2024-09-13T17:45:26.866583+0000","last_active":"2024-09-13T17:45:26.875556+0000","last_peered":"2024-09-13T17:45:26.875556+0000","last_clean":"2024-09-13T17:45:26.875556+0000","last_became_active":"2024-09-13T17:45:26.865997+0000","last_became_peered":"2024-09-13T17:45:26.865997+0000","last_unstale":"2024-09-13T17:45:26.875556+0000","last_undegraded":"2024-09-13T17:45:26.875556+0000","last_fullsized":"2024-09-13T17:45:26.875556+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:26.076527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875645+0000","last_change":"2024-09-13T17:45:26.470813+0000","last_active":"2024-09-13T17:45:26.875645+0000","last_peered":"2024-09-13T17:45:26.875645+0000","last_clean":"2024-09-13T17:45:26.875645+0000","last_became_active":"2024-09-13T17:45:26.470389+0000","last_became_peered":"2024-09-13T17:45:26.470389+0000","last_unstale":"2024-09-13T17:45:26.875645+0000","last_undegraded":"2024-09-13T17:45:26.875645+0000","last_fullsized":"2024-09-13T17:45:26.875645+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:14.941728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859601+0000","last_change":"2024-09-13T17:45:26.856407+0000","last_active":"2024-09-13T17:45:26.859601+0000","last_peered":"2024-09-13T17:45:26.859601+0000","last_clean":"2024-09-13T17:45:26.859601+0000","last_became_active":"2024-09-13T17:45:26.855265+0000","last_became_peered":"2024-09-13T17:45:26.855265+0000","last_unstale":"2024-09-13T17:45:26.859601+0000","last_undegraded":"2024-09-13T17:45:26.859601+0000","last_fullsized":"2024-09-13T17:45:26.859601+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:38.510250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.860242+0000","last_change":"2024-09-13T17:45:26.485330+0000","last_active":"2024-09-13T17:45:26.860242+0000","last_peered":"2024-09-13T17:45:26.860242+0000","last_clean":"2024-09-13T17:45:26.860242+0000","last_became_active":"2024-09-13T17:45:26.483742+0000","last_became_peered":"2024-09-13T17:45:26.483742+0000","last_unstale":"2024-09-13T17:45:26.860242+0000","last_undegraded":"2024-09-13T17:45:26.860242+0000","last_fullsized":"2024-09-13T17:45:26.860242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:36.396235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877222+0000","last_change":"2024-09-13T17:45:26.866163+0000","last_active":"2024-09-13T17:45:26.877222+0000","last_peered":"2024-09-13T17:45:26.877222+0000","last_clean":"2024-09-13T17:45:26.877222+0000","last_became_active":"2024-09-13T17:45:26.863234+0000","last_became_peered":"2024-09-13T17:45:26.863234+0000","last_unstale":"2024-09-13T17:45:26.877222+0000","last_undegraded":"2024-09-13T17:45:26.877222+0000","last_fullsized":"2024-09-13T17:45:26.877222+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:05.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881508+0000","last_change":"2024-09-13T17:45:26.476448+0000","last_active":"2024-09-13T17:45:26.881508+0000","last_peered":"2024-09-13T17:45:26.881508+0000","last_clean":"2024-09-13T17:45:26.881508+0000","last_became_active":"2024-09-13T17:45:26.476065+0000","last_became_peered":"2024-09-13T17:45:26.476065+0000","last_unstale":"2024-09-13T17:45:26.881508+0000","last_undegraded":"2024-09-13T17:45:26.881508+0000","last_fullsized":"2024-09-13T17:45:26.881508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:09.312668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859651+0000","last_change":"2024-09-13T17:45:26.855921+0000","last_active":"2024-09-13T17:45:26.859651+0000","last_peered":"2024-09-13T17:45:26.859651+0000","last_clean":"2024-09-13T17:45:26.859651+0000","last_became_active":"2024-09-13T17:45:26.855424+0000","last_became_peered":"2024-09-13T17:45:26.855424+0000","last_unstale":"2024-09-13T17:45:26.859651+0000","last_undegraded":"2024-09-13T17:45:26.859651+0000","last_fullsized":"2024-09-13T17:45:26.859651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:50.343310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877325+0000","last_change":"2024-09-13T17:45:26.465677+0000","last_active":"2024-09-13T17:45:26.877325+0000","last_peered":"2024-09-13T17:45:26.877325+0000","last_clean":"2024-09-13T17:45:26.877325+0000","last_became_active":"2024-09-13T17:45:26.464625+0000","last_became_peered":"2024-09-13T17:45:26.464625+0000","last_unstale":"2024-09-13T17:45:26.877325+0000","last_undegraded":"2024-09-13T17:45:26.877325+0000","last_fullsized":"2024-09-13T17:45:26.877325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:29.441415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859155+0000","last_change":"2024-09-13T17:45:26.850539+0000","last_active":"2024-09-13T17:45:26.859155+0000","last_peered":"2024-09-13T17:45:26.859155+0000","last_clean":"2024-09-13T17:45:26.859155+0000","last_became_active":"2024-09-13T17:45:26.849631+0000","last_became_peered":"2024-09-13T17:45:26.849631+0000","last_unstale":"2024-09-13T17:45:26.859155+0000","last_undegraded":"2024-09-13T17:45:26.859155+0000","last_fullsized":"2024-09-13T17:45:26.859155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:39.363449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875721+0000","last_change":"2024-09-13T17:45:26.466423+0000","last_active":"2024-09-13T17:45:26.875721+0000","last_peered":"2024-09-13T17:45:26.875721+0000","last_clean":"2024-09-13T17:45:26.875721+0000","last_became_active":"2024-09-13T17:45:26.465264+0000","last_became_peered":"2024-09-13T17:45:26.465264+0000","last_unstale":"2024-09-13T17:45:26.875721+0000","last_undegraded":"2024-09-13T17:45:26.875721+0000","last_fullsized":"2024-09-13T17:45:26.875721+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:17.357306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884238+0000","last_change":"2024-09-13T17:45:26.877910+0000","last_active":"2024-09-13T17:45:26.884238+0000","last_peered":"2024-09-13T17:45:26.884238+0000","last_clean":"2024-09-13T17:45:26.884238+0000","last_became_active":"2024-09-13T17:45:26.877022+0000","last_became_peered":"2024-09-13T17:45:26.877022+0000","last_unstale":"2024-09-13T17:45:26.884238+0000","last_undegraded":"2024-09-13T17:45:26.884238+0000","last_fullsized":"2024-09-13T17:45:26.884238+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:21.495952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876598+0000","last_change":"2024-09-13T17:45:26.482402+0000","last_active":"2024-09-13T17:45:26.876598+0000","last_peered":"2024-09-13T17:45:26.876598+0000","last_clean":"2024-09-13T17:45:26.876598+0000","last_became_active":"2024-09-13T17:45:26.481148+0000","last_became_peered":"2024-09-13T17:45:26.481148+0000","last_unstale":"2024-09-13T17:45:26.876598+0000","last_undegraded":"2024-09-13T17:45:26.876598+0000","last_fullsized":"2024-09-13T17:45:26.876598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:25.050197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879505+0000","last_change":"2024-09-13T17:45:26.868558+0000","last_active":"2024-09-13T17:45:26.879505+0000","last_peered":"2024-09-13T17:45:26.879505+0000","last_clean":"2024-09-13T17:45:26.879505+0000","last_became_active":"2024-09-13T17:45:26.868051+0000","last_became_peered":"2024-09-13T17:45:26.868051+0000","last_unstale":"2024-09-13T17:45:26.879505+0000","last_undegraded":"2024-09-13T17:45:26.879505+0000","last_fullsized":"2024-09-13T17:45:26.879505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:59.853966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.847131+0000","last_change":"2024-09-13T17:45:26.462188+0000","last_active":"2024-09-13T17:45:26.847131+0000","last_peered":"2024-09-13T17:45:26.847131+0000","last_clean":"2024-09-13T17:45:26.847131+0000","last_became_active":"2024-09-13T17:45:26.461671+0000","last_became_peered":"2024-09-13T17:45:26.461671+0000","last_unstale":"2024-09-13T17:45:26.847131+0000","last_undegraded":"2024-09-13T17:45:26.847131+0000","last_fullsized":"2024-09-13T17:45:26.847131+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.822379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865357+0000","last_change":"2024-09-13T17:45:26.860578+0000","last_active":"2024-09-13T17:45:26.865357+0000","last_peered":"2024-09-13T17:45:26.865357+0000","last_clean":"2024-09-13T17:45:26.865357+0000","last_became_active":"2024-09-13T17:45:26.860309+0000","last_became_peered":"2024-09-13T17:45:26.860309+0000","last_unstale":"2024-09-13T17:45:26.865357+0000","last_undegraded":"2024-09-13T17:45:26.865357+0000","last_fullsized":"2024-09-13T17:45:26.865357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:57.480806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.838045+0000","last_change":"2024-09-13T17:45:26.473990+0000","last_active":"2024-09-13T17:45:26.838045+0000","last_peered":"2024-09-13T17:45:26.838045+0000","last_clean":"2024-09-13T17:45:26.838045+0000","last_became_active":"2024-09-13T17:45:26.468331+0000","last_became_peered":"2024-09-13T17:45:26.468331+0000","last_unstale":"2024-09-13T17:45:26.838045+0000","last_undegraded":"2024-09-13T17:45:26.838045+0000","last_fullsized":"2024-09-13T17:45:26.838045+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:43.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875500+0000","last_change":"2024-09-13T17:45:26.866392+0000","last_active":"2024-09-13T17:45:26.875500+0000","last_peered":"2024-09-13T17:45:26.875500+0000","last_clean":"2024-09-13T17:45:26.875500+0000","last_became_active":"2024-09-13T17:45:26.862928+0000","last_became_peered":"2024-09-13T17:45:26.862928+0000","last_unstale":"2024-09-13T17:45:26.875500+0000","last_undegraded":"2024-09-13T17:45:26.875500+0000","last_fullsized":"2024-09-13T17:45:26.875500+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:30.421111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842429+0000","last_change":"2024-09-13T17:45:26.476448+0000","last_active":"2024-09-13T17:45:26.842429+0000","last_peered":"2024-09-13T17:45:26.842429+0000","last_clean":"2024-09-13T17:45:26.842429+0000","last_became_active":"2024-09-13T17:45:26.475617+0000","last_became_peered":"2024-09-13T17:45:26.475617+0000","last_unstale":"2024-09-13T17:45:26.842429+0000","last_undegraded":"2024-09-13T17:45:26.842429+0000","last_fullsized":"2024-09-13T17:45:26.842429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:22.785469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859583+0000","last_change":"2024-09-13T17:45:26.856239+0000","last_active":"2024-09-13T17:45:26.859583+0000","last_peered":"2024-09-13T17:45:26.859583+0000","last_clean":"2024-09-13T17:45:26.859583+0000","last_became_active":"2024-09-13T17:45:26.855030+0000","last_became_peered":"2024-09-13T17:45:26.855030+0000","last_unstale":"2024-09-13T17:45:26.859583+0000","last_undegraded":"2024-09-13T17:45:26.859583+0000","last_fullsized":"2024-09-13T17:45:26.859583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:42.690789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.850967+0000","last_change":"2024-09-13T17:45:26.482572+0000","last_active":"2024-09-13T17:45:26.850967+0000","last_peered":"2024-09-13T17:45:26.850967+0000","last_clean":"2024-09-13T17:45:26.850967+0000","last_became_active":"2024-09-13T17:45:26.481852+0000","last_became_peered":"2024-09-13T17:45:26.481852+0000","last_unstale":"2024-09-13T17:45:26.850967+0000","last_undegraded":"2024-09-13T17:45:26.850967+0000","last_fullsized":"2024-09-13T17:45:26.850967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:04.925857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877201+0000","last_change":"2024-09-13T17:45:26.865172+0000","last_active":"2024-09-13T17:45:26.877201+0000","last_peered":"2024-09-13T17:45:26.877201+0000","last_clean":"2024-09-13T17:45:26.877201+0000","last_became_active":"2024-09-13T17:45:26.862931+0000","last_became_peered":"2024-09-13T17:45:26.862931+0000","last_unstale":"2024-09-13T17:45:26.877201+0000","last_undegraded":"2024-09-13T17:45:26.877201+0000","last_fullsized":"2024-09-13T17:45:26.877201+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:33.221904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859520+0000","last_change":"2024-09-13T17:45:26.475217+0000","last_active":"2024-09-13T17:45:26.859520+0000","last_peered":"2024-09-13T17:45:26.859520+0000","last_clean":"2024-09-13T17:45:26.859520+0000","last_became_active":"2024-09-13T17:45:26.474944+0000","last_became_peered":"2024-09-13T17:45:26.474944+0000","last_unstale":"2024-09-13T17:45:26.859520+0000","last_undegraded":"2024-09-13T17:45:26.859520+0000","last_fullsized":"2024-09-13T17:45:26.859520+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:32.426871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884199+0000","last_change":"2024-09-13T17:45:26.877638+0000","last_active":"2024-09-13T17:45:26.884199+0000","last_peered":"2024-09-13T17:45:26.884199+0000","last_clean":"2024-09-13T17:45:26.884199+0000","last_became_active":"2024-09-13T17:45:26.877062+0000","last_became_peered":"2024-09-13T17:45:26.877062+0000","last_unstale":"2024-09-13T17:45:26.884199+0000","last_undegraded":"2024-09-13T17:45:26.884199+0000","last_fullsized":"2024-09-13T17:45:26.884199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:43.965608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.886860+0000","last_change":"2024-09-13T17:45:26.466199+0000","last_active":"2024-09-13T17:45:26.886860+0000","last_peered":"2024-09-13T17:45:26.886860+0000","last_clean":"2024-09-13T17:45:26.886860+0000","last_became_active":"2024-09-13T17:45:26.464106+0000","last_became_peered":"2024-09-13T17:45:26.464106+0000","last_unstale":"2024-09-13T17:45:26.886860+0000","last_undegraded":"2024-09-13T17:45:26.886860+0000","last_fullsized":"2024-09-13T17:45:26.886860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:50.667743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.874998+0000","last_change":"2024-09-13T17:45:26.862455+0000","last_active":"2024-09-13T17:45:26.874998+0000","last_peered":"2024-09-13T17:45:26.874998+0000","last_clean":"2024-09-13T17:45:26.874998+0000","last_became_active":"2024-09-13T17:45:26.861182+0000","last_became_peered":"2024-09-13T17:45:26.861182+0000","last_unstale":"2024-09-13T17:45:26.874998+0000","last_undegraded":"2024-09-13T17:45:26.874998+0000","last_fullsized":"2024-09-13T17:45:26.874998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:35.493081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877761+0000","last_change":"2024-09-13T17:45:26.468317+0000","last_active":"2024-09-13T17:45:26.877761+0000","last_peered":"2024-09-13T17:45:26.877761+0000","last_clean":"2024-09-13T17:45:26.877761+0000","last_became_active":"2024-09-13T17:45:26.460439+0000","last_became_peered":"2024-09-13T17:45:26.460439+0000","last_unstale":"2024-09-13T17:45:26.877761+0000","last_undegraded":"2024-09-13T17:45:26.877761+0000","last_fullsized":"2024-09-13T17:45:26.877761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881430+0000","last_change":"2024-09-13T17:45:26.878107+0000","last_active":"2024-09-13T17:45:26.881430+0000","last_peered":"2024-09-13T17:45:26.881430+0000","last_clean":"2024-09-13T17:45:26.881430+0000","last_became_active":"2024-09-13T17:45:26.877844+0000","last_became_peered":"2024-09-13T17:45:26.877844+0000","last_unstale":"2024-09-13T17:45:26.881430+0000","last_undegraded":"2024-09-13T17:45:26.881430+0000","last_fullsized":"2024-09-13T17:45:26.881430+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:45.537512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846173+0000","last_change":"2024-09-13T17:45:26.466173+0000","last_active":"2024-09-13T17:45:26.846173+0000","last_peered":"2024-09-13T17:45:26.846173+0000","last_clean":"2024-09-13T17:45:26.846173+0000","last_became_active":"2024-09-13T17:45:26.464683+0000","last_became_peered":"2024-09-13T17:45:26.464683+0000","last_unstale":"2024-09-13T17:45:26.846173+0000","last_undegraded":"2024-09-13T17:45:26.846173+0000","last_fullsized":"2024-09-13T17:45:26.846173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:17:23.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859013+0000","last_change":"2024-09-13T17:45:26.853330+0000","last_active":"2024-09-13T17:45:26.859013+0000","last_peered":"2024-09-13T17:45:26.859013+0000","last_clean":"2024-09-13T17:45:26.859013+0000","last_became_active":"2024-09-13T17:45:26.852784+0000","last_became_peered":"2024-09-13T17:45:26.852784+0000","last_unstale":"2024-09-13T17:45:26.859013+0000","last_undegraded":"2024-09-13T17:45:26.859013+0000","last_fullsized":"2024-09-13T17:45:26.859013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:02.152157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842223+0000","last_change":"2024-09-13T17:45:26.485776+0000","last_active":"2024-09-13T17:45:26.842223+0000","last_peered":"2024-09-13T17:45:26.842223+0000","last_clean":"2024-09-13T17:45:26.842223+0000","last_became_active":"2024-09-13T17:45:26.485473+0000","last_became_peered":"2024-09-13T17:45:26.485473+0000","last_unstale":"2024-09-13T17:45:26.842223+0000","last_undegraded":"2024-09-13T17:45:26.842223+0000","last_fullsized":"2024-09-13T17:45:26.842223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879155+0000","last_change":"2024-09-13T17:45:26.870500+0000","last_active":"2024-09-13T17:45:26.879155+0000","last_peered":"2024-09-13T17:45:26.879155+0000","last_clean":"2024-09-13T17:45:26.879155+0000","last_became_active":"2024-09-13T17:45:26.870245+0000","last_became_peered":"2024-09-13T17:45:26.870245+0000","last_unstale":"2024-09-13T17:45:26.879155+0000","last_undegraded":"2024-09-13T17:45:26.879155+0000","last_fullsized":"2024-09-13T17:45:26.879155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:40.137656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879874+0000","last_change":"2024-09-13T17:45:26.464589+0000","last_active":"2024-09-13T17:45:26.879874+0000","last_peered":"2024-09-13T17:45:26.879874+0000","last_clean":"2024-09-13T17:45:26.879874+0000","last_became_active":"2024-09-13T17:45:26.463835+0000","last_became_peered":"2024-09-13T17:45:26.463835+0000","last_unstale":"2024-09-13T17:45:26.879874+0000","last_undegraded":"2024-09-13T17:45:26.879874+0000","last_fullsized":"2024-09-13T17:45:26.879874+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:18.186378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875581+0000","last_change":"2024-09-13T17:45:26.868705+0000","last_active":"2024-09-13T17:45:26.875581+0000","last_peered":"2024-09-13T17:45:26.875581+0000","last_clean":"2024-09-13T17:45:26.875581+0000","last_became_active":"2024-09-13T17:45:26.866216+0000","last_became_peered":"2024-09-13T17:45:26.866216+0000","last_unstale":"2024-09-13T17:45:26.875581+0000","last_undegraded":"2024-09-13T17:45:26.875581+0000","last_fullsized":"2024-09-13T17:45:26.875581+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:10.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872859+0000","last_change":"2024-09-13T17:45:26.470652+0000","last_active":"2024-09-13T17:45:26.872859+0000","last_peered":"2024-09-13T17:45:26.872859+0000","last_clean":"2024-09-13T17:45:26.872859+0000","last_became_active":"2024-09-13T17:45:26.470068+0000","last_became_peered":"2024-09-13T17:45:26.470068+0000","last_unstale":"2024-09-13T17:45:26.872859+0000","last_undegraded":"2024-09-13T17:45:26.872859+0000","last_fullsized":"2024-09-13T17:45:26.872859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:02.385141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884083+0000","last_change":"2024-09-13T17:45:26.876751+0000","last_active":"2024-09-13T17:45:26.884083+0000","last_peered":"2024-09-13T17:45:26.884083+0000","last_clean":"2024-09-13T17:45:26.884083+0000","last_became_active":"2024-09-13T17:45:26.876443+0000","last_became_peered":"2024-09-13T17:45:26.876443+0000","last_unstale":"2024-09-13T17:45:26.884083+0000","last_undegraded":"2024-09-13T17:45:26.884083+0000","last_fullsized":"2024-09-13T17:45:26.884083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:18.186744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872013+0000","last_change":"2024-09-13T17:45:26.465582+0000","last_active":"2024-09-13T17:45:26.872013+0000","last_peered":"2024-09-13T17:45:26.872013+0000","last_clean":"2024-09-13T17:45:26.872013+0000","last_became_active":"2024-09-13T17:45:26.465082+0000","last_became_peered":"2024-09-13T17:45:26.465082+0000","last_unstale":"2024-09-13T17:45:26.872013+0000","last_undegraded":"2024-09-13T17:45:26.872013+0000","last_fullsized":"2024-09-13T17:45:26.872013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.048020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875418+0000","last_change":"2024-09-13T17:45:26.864827+0000","last_active":"2024-09-13T17:45:26.875418+0000","last_peered":"2024-09-13T17:45:26.875418+0000","last_clean":"2024-09-13T17:45:26.875418+0000","last_became_active":"2024-09-13T17:45:26.861456+0000","last_became_peered":"2024-09-13T17:45:26.861456+0000","last_unstale":"2024-09-13T17:45:26.875418+0000","last_undegraded":"2024-09-13T17:45:26.875418+0000","last_fullsized":"2024-09-13T17:45:26.875418+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:19.479096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846501+0000","last_change":"2024-09-13T17:45:26.466500+0000","last_active":"2024-09-13T17:45:26.846501+0000","last_peered":"2024-09-13T17:45:26.846501+0000","last_clean":"2024-09-13T17:45:26.846501+0000","last_became_active":"2024-09-13T17:45:26.465685+0000","last_became_peered":"2024-09-13T17:45:26.465685+0000","last_unstale":"2024-09-13T17:45:26.846501+0000","last_undegraded":"2024-09-13T17:45:26.846501+0000","last_fullsized":"2024-09-13T17:45:26.846501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:28.225692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863832+0000","last_change":"2024-09-13T17:45:26.858987+0000","last_active":"2024-09-13T17:45:26.863832+0000","last_peered":"2024-09-13T17:45:26.863832+0000","last_clean":"2024-09-13T17:45:26.863832+0000","last_became_active":"2024-09-13T17:45:26.858742+0000","last_became_peered":"2024-09-13T17:45:26.858742+0000","last_unstale":"2024-09-13T17:45:26.863832+0000","last_undegraded":"2024-09-13T17:45:26.863832+0000","last_fullsized":"2024-09-13T17:45:26.863832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875791+0000","last_change":"2024-09-13T17:45:26.471744+0000","last_active":"2024-09-13T17:45:26.875791+0000","last_peered":"2024-09-13T17:45:26.875791+0000","last_clean":"2024-09-13T17:45:26.875791+0000","last_became_active":"2024-09-13T17:45:26.471366+0000","last_became_peered":"2024-09-13T17:45:26.471366+0000","last_unstale":"2024-09-13T17:45:26.875791+0000","last_undegraded":"2024-09-13T17:45:26.875791+0000","last_fullsized":"2024-09-13T17:45:26.875791+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:47.021350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884226+0000","last_change":"2024-09-13T17:45:26.877656+0000","last_active":"2024-09-13T17:45:26.884226+0000","last_peered":"2024-09-13T17:45:26.884226+0000","last_clean":"2024-09-13T17:45:26.884226+0000","last_became_active":"2024-09-13T17:45:26.877306+0000","last_became_peered":"2024-09-13T17:45:26.877306+0000","last_unstale":"2024-09-13T17:45:26.884226+0000","last_undegraded":"2024-09-13T17:45:26.884226+0000","last_fullsized":"2024-09-13T17:45:26.884226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:15.557735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841766+0000","last_change":"2024-09-13T17:45:26.482746+0000","last_active":"2024-09-13T17:45:26.841766+0000","last_peered":"2024-09-13T17:45:26.841766+0000","last_clean":"2024-09-13T17:45:26.841766+0000","last_became_active":"2024-09-13T17:45:26.482078+0000","last_became_peered":"2024-09-13T17:45:26.482078+0000","last_unstale":"2024-09-13T17:45:26.841766+0000","last_undegraded":"2024-09-13T17:45:26.841766+0000","last_fullsized":"2024-09-13T17:45:26.841766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:45.524332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881298+0000","last_change":"2024-09-13T17:45:26.875733+0000","last_active":"2024-09-13T17:45:26.881298+0000","last_peered":"2024-09-13T17:45:26.881298+0000","last_clean":"2024-09-13T17:45:26.881298+0000","last_became_active":"2024-09-13T17:45:26.875464+0000","last_became_peered":"2024-09-13T17:45:26.875464+0000","last_unstale":"2024-09-13T17:45:26.881298+0000","last_undegraded":"2024-09-13T17:45:26.881298+0000","last_fullsized":"2024-09-13T17:45:26.881298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:02.152157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842326+0000","last_change":"2024-09-13T17:45:26.485613+0000","last_active":"2024-09-13T17:45:26.842326+0000","last_peered":"2024-09-13T17:45:26.842326+0000","last_clean":"2024-09-13T17:45:26.842326+0000","last_became_active":"2024-09-13T17:45:26.485333+0000","last_became_peered":"2024-09-13T17:45:26.485333+0000","last_unstale":"2024-09-13T17:45:26.842326+0000","last_undegraded":"2024-09-13T17:45:26.842326+0000","last_fullsized":"2024-09-13T17:45:26.842326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:16.875343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875471+0000","last_change":"2024-09-13T17:45:26.866094+0000","last_active":"2024-09-13T17:45:26.875471+0000","last_peered":"2024-09-13T17:45:26.875471+0000","last_clean":"2024-09-13T17:45:26.875471+0000","last_became_active":"2024-09-13T17:45:26.862870+0000","last_became_peered":"2024-09-13T17:45:26.862870+0000","last_unstale":"2024-09-13T17:45:26.875471+0000","last_undegraded":"2024-09-13T17:45:26.875471+0000","last_fullsized":"2024-09-13T17:45:26.875471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:12.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841265+0000","last_change":"2024-09-13T17:45:26.481550+0000","last_active":"2024-09-13T17:45:26.841265+0000","last_peered":"2024-09-13T17:45:26.841265+0000","last_clean":"2024-09-13T17:45:26.841265+0000","last_became_active":"2024-09-13T17:45:26.481128+0000","last_became_peered":"2024-09-13T17:45:26.481128+0000","last_unstale":"2024-09-13T17:45:26.841265+0000","last_undegraded":"2024-09-13T17:45:26.841265+0000","last_fullsized":"2024-09-13T17:45:26.841265+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:24:59.217212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.874938+0000","last_change":"2024-09-13T17:45:26.863363+0000","last_active":"2024-09-13T17:45:26.874938+0000","last_peered":"2024-09-13T17:45:26.874938+0000","last_clean":"2024-09-13T17:45:26.874938+0000","last_became_active":"2024-09-13T17:45:26.862490+0000","last_became_peered":"2024-09-13T17:45:26.862490+0000","last_unstale":"2024-09-13T17:45:26.874938+0000","last_undegraded":"2024-09-13T17:45:26.874938+0000","last_fullsized":"2024-09-13T17:45:26.874938+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:06.864302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.858402+0000","last_change":"2024-09-13T17:45:26.476236+0000","last_active":"2024-09-13T17:45:26.858402+0000","last_peered":"2024-09-13T17:45:26.858402+0000","last_clean":"2024-09-13T17:45:26.858402+0000","last_became_active":"2024-09-13T17:45:26.474740+0000","last_became_peered":"2024-09-13T17:45:26.474740+0000","last_unstale":"2024-09-13T17:45:26.858402+0000","last_undegraded":"2024-09-13T17:45:26.858402+0000","last_fullsized":"2024-09-13T17:45:26.858402+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:34:05.270907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881373+0000","last_change":"2024-09-13T17:45:26.873825+0000","last_active":"2024-09-13T17:45:26.881373+0000","last_peered":"2024-09-13T17:45:26.881373+0000","last_clean":"2024-09-13T17:45:26.881373+0000","last_became_active":"2024-09-13T17:45:26.873383+0000","last_became_peered":"2024-09-13T17:45:26.873383+0000","last_unstale":"2024-09-13T17:45:26.881373+0000","last_undegraded":"2024-09-13T17:45:26.881373+0000","last_fullsized":"2024-09-13T17:45:26.881373+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:33.687155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841901+0000","last_change":"2024-09-13T17:45:26.468248+0000","last_active":"2024-09-13T17:45:26.841901+0000","last_peered":"2024-09-13T17:45:26.841901+0000","last_clean":"2024-09-13T17:45:26.841901+0000","last_became_active":"2024-09-13T17:45:26.466994+0000","last_became_peered":"2024-09-13T17:45:26.466994+0000","last_unstale":"2024-09-13T17:45:26.841901+0000","last_undegraded":"2024-09-13T17:45:26.841901+0000","last_fullsized":"2024-09-13T17:45:26.841901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:42.730586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877262+0000","last_change":"2024-09-13T17:45:26.866560+0000","last_active":"2024-09-13T17:45:26.877262+0000","last_peered":"2024-09-13T17:45:26.877262+0000","last_clean":"2024-09-13T17:45:26.877262+0000","last_became_active":"2024-09-13T17:45:26.864707+0000","last_became_peered":"2024-09-13T17:45:26.864707+0000","last_unstale":"2024-09-13T17:45:26.877262+0000","last_undegraded":"2024-09-13T17:45:26.877262+0000","last_fullsized":"2024-09-13T17:45:26.877262+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:35.606328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846766+0000","last_change":"2024-09-13T17:45:26.475689+0000","last_active":"2024-09-13T17:45:26.846766+0000","last_peered":"2024-09-13T17:45:26.846766+0000","last_clean":"2024-09-13T17:45:26.846766+0000","last_became_active":"2024-09-13T17:45:26.470568+0000","last_became_peered":"2024-09-13T17:45:26.470568+0000","last_unstale":"2024-09-13T17:45:26.846766+0000","last_undegraded":"2024-09-13T17:45:26.846766+0000","last_fullsized":"2024-09-13T17:45:26.846766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:51.371173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884157+0000","last_change":"2024-09-13T17:45:26.877467+0000","last_active":"2024-09-13T17:45:26.884157+0000","last_peered":"2024-09-13T17:45:26.884157+0000","last_clean":"2024-09-13T17:45:26.884157+0000","last_became_active":"2024-09-13T17:45:26.876694+0000","last_became_peered":"2024-09-13T17:45:26.876694+0000","last_unstale":"2024-09-13T17:45:26.884157+0000","last_undegraded":"2024-09-13T17:45:26.884157+0000","last_fullsized":"2024-09-13T17:45:26.884157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:09.564935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.847759+0000","last_change":"2024-09-13T17:45:26.483859+0000","last_active":"2024-09-13T17:45:26.847759+0000","last_peered":"2024-09-13T17:45:26.847759+0000","last_clean":"2024-09-13T17:45:26.847759+0000","last_became_active":"2024-09-13T17:45:26.483529+0000","last_became_peered":"2024-09-13T17:45:26.483529+0000","last_unstale":"2024-09-13T17:45:26.847759+0000","last_undegraded":"2024-09-13T17:45:26.847759+0000","last_fullsized":"2024-09-13T17:45:26.847759+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:07.902010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859178+0000","last_change":"2024-09-13T17:45:26.850728+0000","last_active":"2024-09-13T17:45:26.859178+0000","last_peered":"2024-09-13T17:45:26.859178+0000","last_clean":"2024-09-13T17:45:26.859178+0000","last_became_active":"2024-09-13T17:45:26.849950+0000","last_became_peered":"2024-09-13T17:45:26.849950+0000","last_unstale":"2024-09-13T17:45:26.859178+0000","last_undegraded":"2024-09-13T17:45:26.859178+0000","last_fullsized":"2024-09-13T17:45:26.859178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:27:37.261491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846859+0000","last_change":"2024-09-13T17:45:26.471586+0000","last_active":"2024-09-13T17:45:26.846859+0000","last_peered":"2024-09-13T17:45:26.846859+0000","last_clean":"2024-09-13T17:45:26.846859+0000","last_became_active":"2024-09-13T17:45:26.471071+0000","last_became_peered":"2024-09-13T17:45:26.471071+0000","last_unstale":"2024-09-13T17:45:26.846859+0000","last_undegraded":"2024-09-13T17:45:26.846859+0000","last_fullsized":"2024-09-13T17:45:26.846859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:51.549791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875593+0000","last_change":"2024-09-13T17:45:26.864664+0000","last_active":"2024-09-13T17:45:26.875593+0000","last_peered":"2024-09-13T17:45:26.875593+0000","last_clean":"2024-09-13T17:45:26.875593+0000","last_became_active":"2024-09-13T17:45:26.862643+0000","last_became_peered":"2024-09-13T17:45:26.862643+0000","last_unstale":"2024-09-13T17:45:26.875593+0000","last_undegraded":"2024-09-13T17:45:26.875593+0000","last_fullsized":"2024-09-13T17:45:26.875593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:14.923370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841818+0000","last_change":"2024-09-13T17:45:26.482889+0000","last_active":"2024-09-13T17:45:26.841818+0000","last_peered":"2024-09-13T17:45:26.841818+0000","last_clean":"2024-09-13T17:45:26.841818+0000","last_became_active":"2024-09-13T17:45:26.482143+0000","last_became_peered":"2024-09-13T17:45:26.482143+0000","last_unstale":"2024-09-13T17:45:26.841818+0000","last_undegraded":"2024-09-13T17:45:26.841818+0000","last_fullsized":"2024-09-13T17:45:26.841818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:18.594276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859032+0000","last_change":"2024-09-13T17:45:26.853408+0000","last_active":"2024-09-13T17:45:26.859032+0000","last_peered":"2024-09-13T17:45:26.859032+0000","last_clean":"2024-09-13T17:45:26.859032+0000","last_became_active":"2024-09-13T17:45:26.853053+0000","last_became_peered":"2024-09-13T17:45:26.853053+0000","last_unstale":"2024-09-13T17:45:26.859032+0000","last_undegraded":"2024-09-13T17:45:26.859032+0000","last_fullsized":"2024-09-13T17:45:26.859032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:19.666501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.837493+0000","last_change":"2024-09-13T17:45:26.460266+0000","last_active":"2024-09-13T17:45:26.837493+0000","last_peered":"2024-09-13T17:45:26.837493+0000","last_clean":"2024-09-13T17:45:26.837493+0000","last_became_active":"2024-09-13T17:45:26.459918+0000","last_became_peered":"2024-09-13T17:45:26.459918+0000","last_unstale":"2024-09-13T17:45:26.837493+0000","last_undegraded":"2024-09-13T17:45:26.837493+0000","last_fullsized":"2024-09-13T17:45:26.837493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:43.446072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.858810+0000","last_change":"2024-09-13T17:45:26.853333+0000","last_active":"2024-09-13T17:45:26.858810+0000","last_peered":"2024-09-13T17:45:26.858810+0000","last_clean":"2024-09-13T17:45:26.858810+0000","last_became_active":"2024-09-13T17:45:26.853073+0000","last_became_peered":"2024-09-13T17:45:26.853073+0000","last_unstale":"2024-09-13T17:45:26.858810+0000","last_undegraded":"2024-09-13T17:45:26.858810+0000","last_fullsized":"2024-09-13T17:45:26.858810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:15.557735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.848208+0000","last_change":"2024-09-13T17:45:26.465422+0000","last_active":"2024-09-13T17:45:26.848208+0000","last_peered":"2024-09-13T17:45:26.848208+0000","last_clean":"2024-09-13T17:45:26.848208+0000","last_became_active":"2024-09-13T17:45:26.464899+0000","last_became_peered":"2024-09-13T17:45:26.464899+0000","last_unstale":"2024-09-13T17:45:26.848208+0000","last_undegraded":"2024-09-13T17:45:26.848208+0000","last_fullsized":"2024-09-13T17:45:26.848208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:04.308482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.868211+0000","last_change":"2024-09-13T17:45:26.867663+0000","last_active":"2024-09-13T17:45:26.868211+0000","last_peered":"2024-09-13T17:45:26.868211+0000","last_clean":"2024-09-13T17:45:26.868211+0000","last_became_active":"2024-09-13T17:45:26.867406+0000","last_became_peered":"2024-09-13T17:45:26.867406+0000","last_unstale":"2024-09-13T17:45:26.868211+0000","last_undegraded":"2024-09-13T17:45:26.868211+0000","last_fullsized":"2024-09-13T17:45:26.868211+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:29:07.999777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859438+0000","last_change":"2024-09-13T17:45:26.475549+0000","last_active":"2024-09-13T17:45:26.859438+0000","last_peered":"2024-09-13T17:45:26.859438+0000","last_clean":"2024-09-13T17:45:26.859438+0000","last_became_active":"2024-09-13T17:45:26.475160+0000","last_became_peered":"2024-09-13T17:45:26.475160+0000","last_unstale":"2024-09-13T17:45:26.859438+0000","last_undegraded":"2024-09-13T17:45:26.859438+0000","last_fullsized":"2024-09-13T17:45:26.859438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:32.768874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"67'1925","reported_seq":2162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:27.533172+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:27.533172+0000","last_peered":"2024-09-13T17:45:27.533172+0000","last_clean":"2024-09-13T17:45:27.533172+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:27.533172+0000","last_undegraded":"2024-09-13T17:45:27.533172+0000","last_fullsized":"2024-09-13T17:45:27.533172+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":1925,"log_dups_size":0,"ondisk_log_size":1925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":223,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":223,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1925,"ondisk_log_size":1925,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738413,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":671744,"data_stored":256771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738414,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1228800,"data_stored":840346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.378}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.41299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738413,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1261568,"data_stored":847013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:30.897 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:31.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b50d82640 1 Processor -- start 2024-09-13T17:45:31.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b50d82640 1 -- start start 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b50d82640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5b4c137d60 0x7f5b4c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b50d82640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4c0a00b0 con 0x7f5b4c138140 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b50d82640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b4c0a12d0 con 0x7f5b4c137d60 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b4a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5b4c137d60 0x7f5b4c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b49d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5b4c138140 0x7f5b4c138590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52300/0 (socket says 172.21.15.96:52300) 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b49d9b640 1 -- 172.21.15.96:0/663782804 learned_addr learned my addr 172.21.15.96:0/663782804 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b4a59c640 1 -- 172.21.15.96:0/663782804 >> v1:172.21.15.96:6789/0 conn(0x7f5b4c138140 legacy=0x7f5b4c138590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.054+0000 7f5b4a59c640 1 -- 172.21.15.96:0/663782804 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c0a0350 con 0x7f5b4c137d60 2024-09-13T17:45:31.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b4a59c640 1 --2- 172.21.15.96:0/663782804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a0d90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad996f23b242734 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b4959a640 1 -- 172.21.15.96:0/663782804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b440a5650 con 0x7f5b4c137d60 2024-09-13T17:45:31.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b4959a640 1 -- 172.21.15.96:0/663782804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b440a57b0 con 0x7f5b4c137d60 2024-09-13T17:45:31.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b4959a640 1 -- 172.21.15.96:0/663782804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b440a5a70 con 0x7f5b4c137d60 2024-09-13T17:45:31.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- 172.21.15.96:0/663782804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 msgr2=0x7f5b4c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 --2- 172.21.15.96:0/663782804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a0d90 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4401c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- 172.21.15.96:0/663782804 shutdown_connections 2024-09-13T17:45:31.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 --2- 172.21.15.96:0/663782804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a0d90 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- 172.21.15.96:0/663782804 >> 172.21.15.96:0/663782804 conn(0x7f5b4c06ef80 msgr2=0x7f5b4c0713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:31.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- 172.21.15.96:0/663782804 shutdown_connections 2024-09-13T17:45:31.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- 172.21.15.96:0/663782804 wait complete. 2024-09-13T17:45:31.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 Processor -- start 2024-09-13T17:45:31.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.058+0000 7f5b50d82640 1 -- start start 2024-09-13T17:45:31.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b50d82640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b50d82640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4c0a4ce0 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b4a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45990/0 (socket says 172.21.15.96:45990) 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b4a59c640 1 -- 172.21.15.96:0/2040776607 learned_addr learned my addr 172.21.15.96:0/2040776607 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b4a59c640 1 -- 172.21.15.96:0/2040776607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44009a70 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b4a59c640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5b4401a510 tx=0x7f5b44019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b440ad800 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b440ad960 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b4c0a81f0 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b440adc40 con 0x7f5b4c137d60 2024-09-13T17:45:31.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4c0a62e0 con 0x7f5b4c137d60 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5b440b4050 con 0x7f5b4c137d60 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4c0a6680 con 0x7f5b4c137d60 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 0x7f5b2c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5b440e2880 con 0x7f5b4c137d60 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b49d9b640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 0x7f5b2c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.062+0000 7f5b49d9b640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 0x7f5b2c0379a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b40001610 tx=0x7f5b40007ad0 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.066+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5b440c1590 con 0x7f5b4c137d60 2024-09-13T17:45:31.215 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:31.214+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:31.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.230+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 --> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5b4c13c4a0 con 0x7f5b2c0354e0 2024-09-13T17:45:31.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.242+0000 7f5b3affd640 1 -- 172.21.15.96:0/2040776607 <== mgr.5260 v2:172.21.15.96:6824/3517445048 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+250147 (secure 0 0 0) 0x7f5b4c13c4a0 con 0x7f5b2c0354e0 2024-09-13T17:45:31.246 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:31.247 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:31.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 msgr2=0x7f5b2c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 0x7f5b2c0379a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5b40001610 tx=0x7f5b40007ad0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 msgr2=0x7f5b4c0a52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f5b4401a510 tx=0x7f5b44019680 comp rx=0 tx=0).stop 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 shutdown_connections 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5b2c0354e0 0x7f5b2c0379a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 --2- 172.21.15.96:0/2040776607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b4c137d60 0x7f5b4c0a52c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 >> 172.21.15.96:0/2040776607 conn(0x7f5b4c06ef80 msgr2=0x7f5b4c1372d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 shutdown_connections 2024-09-13T17:45:31.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.246+0000 7f5b50d82640 1 -- 172.21.15.96:0/2040776607 wait complete. 2024-09-13T17:45:31.265 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T17:45:29.545923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":267,"num_write_kb":1908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1963,"ondisk_log_size":1963,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":77,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":108744,"kb_used_data":3088,"kb_used_omap":4,"kb_used_meta":105595,"kb_avail":283006776,"statfs":{"total":289910292480,"available":289798938624,"internally_reserved":0,"allocated":3162112,"data_stored":1944130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":108129619},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.126420"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865488+0000","last_change":"2024-09-13T17:45:26.857597+0000","last_active":"2024-09-13T17:45:26.865488+0000","last_peered":"2024-09-13T17:45:26.865488+0000","last_clean":"2024-09-13T17:45:26.865488+0000","last_became_active":"2024-09-13T17:45:26.857268+0000","last_became_peered":"2024-09-13T17:45:26.857268+0000","last_unstale":"2024-09-13T17:45:26.865488+0000","last_undegraded":"2024-09-13T17:45:26.865488+0000","last_fullsized":"2024-09-13T17:45:26.865488+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:27.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859531+0000","last_change":"2024-09-13T17:45:26.854681+0000","last_active":"2024-09-13T17:45:26.859531+0000","last_peered":"2024-09-13T17:45:26.859531+0000","last_clean":"2024-09-13T17:45:26.859531+0000","last_became_active":"2024-09-13T17:45:26.854327+0000","last_became_peered":"2024-09-13T17:45:26.854327+0000","last_unstale":"2024-09-13T17:45:26.859531+0000","last_undegraded":"2024-09-13T17:45:26.859531+0000","last_fullsized":"2024-09-13T17:45:26.859531+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:02.521318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865388+0000","last_change":"2024-09-13T17:45:26.860765+0000","last_active":"2024-09-13T17:45:26.865388+0000","last_peered":"2024-09-13T17:45:26.865388+0000","last_clean":"2024-09-13T17:45:26.865388+0000","last_became_active":"2024-09-13T17:45:26.860494+0000","last_became_peered":"2024-09-13T17:45:26.860494+0000","last_unstale":"2024-09-13T17:45:26.865388+0000","last_undegraded":"2024-09-13T17:45:26.865388+0000","last_fullsized":"2024-09-13T17:45:26.865388+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:25:48.120611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865401+0000","last_change":"2024-09-13T17:45:26.860895+0000","last_active":"2024-09-13T17:45:26.865401+0000","last_peered":"2024-09-13T17:45:26.865401+0000","last_clean":"2024-09-13T17:45:26.865401+0000","last_became_active":"2024-09-13T17:45:26.860511+0000","last_became_peered":"2024-09-13T17:45:26.860511+0000","last_unstale":"2024-09-13T17:45:26.865401+0000","last_undegraded":"2024-09-13T17:45:26.865401+0000","last_fullsized":"2024-09-13T17:45:26.865401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:29:05.910353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863437+0000","last_change":"2024-09-13T17:45:26.858688+0000","last_active":"2024-09-13T17:45:26.863437+0000","last_peered":"2024-09-13T17:45:26.863437+0000","last_clean":"2024-09-13T17:45:26.863437+0000","last_became_active":"2024-09-13T17:45:26.858384+0000","last_became_peered":"2024-09-13T17:45:26.858384+0000","last_unstale":"2024-09-13T17:45:26.863437+0000","last_undegraded":"2024-09-13T17:45:26.863437+0000","last_fullsized":"2024-09-13T17:45:26.863437+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:18:53.025921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881207+0000","last_change":"2024-09-13T17:45:26.875078+0000","last_active":"2024-09-13T17:45:26.881207+0000","last_peered":"2024-09-13T17:45:26.881207+0000","last_clean":"2024-09-13T17:45:26.881207+0000","last_became_active":"2024-09-13T17:45:26.874666+0000","last_became_peered":"2024-09-13T17:45:26.874666+0000","last_unstale":"2024-09-13T17:45:26.881207+0000","last_undegraded":"2024-09-13T17:45:26.881207+0000","last_fullsized":"2024-09-13T17:45:26.881207+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:46:16.908325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876147+0000","last_change":"2024-09-13T17:45:26.874893+0000","last_active":"2024-09-13T17:45:26.876147+0000","last_peered":"2024-09-13T17:45:26.876147+0000","last_clean":"2024-09-13T17:45:26.876147+0000","last_became_active":"2024-09-13T17:45:26.874643+0000","last_became_peered":"2024-09-13T17:45:26.874643+0000","last_unstale":"2024-09-13T17:45:26.876147+0000","last_undegraded":"2024-09-13T17:45:26.876147+0000","last_fullsized":"2024-09-13T17:45:26.876147+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:02:36.836974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876673+0000","last_change":"2024-09-13T17:45:26.868970+0000","last_active":"2024-09-13T17:45:26.876673+0000","last_peered":"2024-09-13T17:45:26.876673+0000","last_clean":"2024-09-13T17:45:26.876673+0000","last_became_active":"2024-09-13T17:45:26.868681+0000","last_became_peered":"2024-09-13T17:45:26.868681+0000","last_unstale":"2024-09-13T17:45:26.876673+0000","last_undegraded":"2024-09-13T17:45:26.876673+0000","last_fullsized":"2024-09-13T17:45:26.876673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:36:14.740110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881391+0000","last_change":"2024-09-13T17:45:26.873882+0000","last_active":"2024-09-13T17:45:26.881391+0000","last_peered":"2024-09-13T17:45:26.881391+0000","last_clean":"2024-09-13T17:45:26.881391+0000","last_became_active":"2024-09-13T17:45:26.873611+0000","last_became_peered":"2024-09-13T17:45:26.873611+0000","last_unstale":"2024-09-13T17:45:26.881391+0000","last_undegraded":"2024-09-13T17:45:26.881391+0000","last_fullsized":"2024-09-13T17:45:26.881391+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:21:17.153513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884059+0000","last_change":"2024-09-13T17:45:26.876739+0000","last_active":"2024-09-13T17:45:26.884059+0000","last_peered":"2024-09-13T17:45:26.884059+0000","last_clean":"2024-09-13T17:45:26.884059+0000","last_became_active":"2024-09-13T17:45:26.876433+0000","last_became_peered":"2024-09-13T17:45:26.876433+0000","last_unstale":"2024-09-13T17:45:26.884059+0000","last_undegraded":"2024-09-13T17:45:26.884059+0000","last_fullsized":"2024-09-13T17:45:26.884059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:42.690789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859632+0000","last_change":"2024-09-13T17:45:26.855741+0000","last_active":"2024-09-13T17:45:26.859632+0000","last_peered":"2024-09-13T17:45:26.859632+0000","last_clean":"2024-09-13T17:45:26.859632+0000","last_became_active":"2024-09-13T17:45:26.855060+0000","last_became_peered":"2024-09-13T17:45:26.855060+0000","last_unstale":"2024-09-13T17:45:26.859632+0000","last_undegraded":"2024-09-13T17:45:26.859632+0000","last_fullsized":"2024-09-13T17:45:26.859632+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:28.806397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877242+0000","last_change":"2024-09-13T17:45:26.866296+0000","last_active":"2024-09-13T17:45:26.877242+0000","last_peered":"2024-09-13T17:45:26.877242+0000","last_clean":"2024-09-13T17:45:26.877242+0000","last_became_active":"2024-09-13T17:45:26.863281+0000","last_became_peered":"2024-09-13T17:45:26.863281+0000","last_unstale":"2024-09-13T17:45:26.877242+0000","last_undegraded":"2024-09-13T17:45:26.877242+0000","last_fullsized":"2024-09-13T17:45:26.877242+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:22:58.053260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876190+0000","last_change":"2024-09-13T17:45:26.866601+0000","last_active":"2024-09-13T17:45:26.876190+0000","last_peered":"2024-09-13T17:45:26.876190+0000","last_clean":"2024-09-13T17:45:26.876190+0000","last_became_active":"2024-09-13T17:45:26.866106+0000","last_became_peered":"2024-09-13T17:45:26.866106+0000","last_unstale":"2024-09-13T17:45:26.876190+0000","last_undegraded":"2024-09-13T17:45:26.876190+0000","last_fullsized":"2024-09-13T17:45:26.876190+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:38:17.165403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863813+0000","last_change":"2024-09-13T17:45:26.858852+0000","last_active":"2024-09-13T17:45:26.863813+0000","last_peered":"2024-09-13T17:45:26.863813+0000","last_clean":"2024-09-13T17:45:26.863813+0000","last_became_active":"2024-09-13T17:45:26.858594+0000","last_became_peered":"2024-09-13T17:45:26.858594+0000","last_unstale":"2024-09-13T17:45:26.863813+0000","last_undegraded":"2024-09-13T17:45:26.863813+0000","last_fullsized":"2024-09-13T17:45:26.863813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:35:59.760400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.868170+0000","last_change":"2024-09-13T17:45:26.867404+0000","last_active":"2024-09-13T17:45:26.868170+0000","last_peered":"2024-09-13T17:45:26.868170+0000","last_clean":"2024-09-13T17:45:26.868170+0000","last_became_active":"2024-09-13T17:45:26.867102+0000","last_became_peered":"2024-09-13T17:45:26.867102+0000","last_unstale":"2024-09-13T17:45:26.868170+0000","last_undegraded":"2024-09-13T17:45:26.868170+0000","last_fullsized":"2024-09-13T17:45:26.868170+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:20.032657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872631+0000","last_change":"2024-09-13T17:45:26.868827+0000","last_active":"2024-09-13T17:45:26.872631+0000","last_peered":"2024-09-13T17:45:26.872631+0000","last_clean":"2024-09-13T17:45:26.872631+0000","last_became_active":"2024-09-13T17:45:26.868576+0000","last_became_peered":"2024-09-13T17:45:26.868576+0000","last_unstale":"2024-09-13T17:45:26.872631+0000","last_undegraded":"2024-09-13T17:45:26.872631+0000","last_fullsized":"2024-09-13T17:45:26.872631+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:28.806397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875732+0000","last_change":"2024-09-13T17:45:26.869411+0000","last_active":"2024-09-13T17:45:26.875732+0000","last_peered":"2024-09-13T17:45:26.875732+0000","last_clean":"2024-09-13T17:45:26.875732+0000","last_became_active":"2024-09-13T17:45:26.869158+0000","last_became_peered":"2024-09-13T17:45:26.869158+0000","last_unstale":"2024-09-13T17:45:26.875732+0000","last_undegraded":"2024-09-13T17:45:26.875732+0000","last_fullsized":"2024-09-13T17:45:26.875732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:09:33.306502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884213+0000","last_change":"2024-09-13T17:45:26.877716+0000","last_active":"2024-09-13T17:45:26.884213+0000","last_peered":"2024-09-13T17:45:26.884213+0000","last_clean":"2024-09-13T17:45:26.884213+0000","last_became_active":"2024-09-13T17:45:26.876640+0000","last_became_peered":"2024-09-13T17:45:26.876640+0000","last_unstale":"2024-09-13T17:45:26.884213+0000","last_undegraded":"2024-09-13T17:45:26.884213+0000","last_fullsized":"2024-09-13T17:45:26.884213+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:10:02.521318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875798+0000","last_change":"2024-09-13T17:45:26.871016+0000","last_active":"2024-09-13T17:45:26.875798+0000","last_peered":"2024-09-13T17:45:26.875798+0000","last_clean":"2024-09-13T17:45:26.875798+0000","last_became_active":"2024-09-13T17:45:26.864520+0000","last_became_peered":"2024-09-13T17:45:26.864520+0000","last_unstale":"2024-09-13T17:45:26.875798+0000","last_undegraded":"2024-09-13T17:45:26.875798+0000","last_fullsized":"2024-09-13T17:45:26.875798+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:54:11.472571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884041+0000","last_change":"2024-09-13T17:45:26.876378+0000","last_active":"2024-09-13T17:45:26.884041+0000","last_peered":"2024-09-13T17:45:26.884041+0000","last_clean":"2024-09-13T17:45:26.884041+0000","last_became_active":"2024-09-13T17:45:26.875700+0000","last_became_peered":"2024-09-13T17:45:26.875700+0000","last_unstale":"2024-09-13T17:45:26.884041+0000","last_undegraded":"2024-09-13T17:45:26.884041+0000","last_fullsized":"2024-09-13T17:45:26.884041+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:27:58.649338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881324+0000","last_change":"2024-09-13T17:45:26.875452+0000","last_active":"2024-09-13T17:45:26.881324+0000","last_peered":"2024-09-13T17:45:26.881324+0000","last_clean":"2024-09-13T17:45:26.881324+0000","last_became_active":"2024-09-13T17:45:26.875203+0000","last_became_peered":"2024-09-13T17:45:26.875203+0000","last_unstale":"2024-09-13T17:45:26.881324+0000","last_undegraded":"2024-09-13T17:45:26.881324+0000","last_fullsized":"2024-09-13T17:45:26.881324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:38.510250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.867552+0000","last_change":"2024-09-13T17:45:26.864287+0000","last_active":"2024-09-13T17:45:26.867552+0000","last_peered":"2024-09-13T17:45:26.867552+0000","last_clean":"2024-09-13T17:45:26.867552+0000","last_became_active":"2024-09-13T17:45:26.863894+0000","last_became_peered":"2024-09-13T17:45:26.863894+0000","last_unstale":"2024-09-13T17:45:26.867552+0000","last_undegraded":"2024-09-13T17:45:26.867552+0000","last_fullsized":"2024-09-13T17:45:26.867552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875535+0000","last_change":"2024-09-13T17:45:26.868513+0000","last_active":"2024-09-13T17:45:26.875535+0000","last_peered":"2024-09-13T17:45:26.875535+0000","last_clean":"2024-09-13T17:45:26.875535+0000","last_became_active":"2024-09-13T17:45:26.864992+0000","last_became_peered":"2024-09-13T17:45:26.864992+0000","last_unstale":"2024-09-13T17:45:26.875535+0000","last_undegraded":"2024-09-13T17:45:26.875535+0000","last_fullsized":"2024-09-13T17:45:26.875535+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:39:33.269069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876119+0000","last_change":"2024-09-13T17:45:26.874888+0000","last_active":"2024-09-13T17:45:26.876119+0000","last_peered":"2024-09-13T17:45:26.876119+0000","last_clean":"2024-09-13T17:45:26.876119+0000","last_became_active":"2024-09-13T17:45:26.874655+0000","last_became_peered":"2024-09-13T17:45:26.874655+0000","last_unstale":"2024-09-13T17:45:26.876119+0000","last_undegraded":"2024-09-13T17:45:26.876119+0000","last_fullsized":"2024-09-13T17:45:26.876119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:50.343310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881348+0000","last_change":"2024-09-13T17:45:26.873644+0000","last_active":"2024-09-13T17:45:26.881348+0000","last_peered":"2024-09-13T17:45:26.881348+0000","last_clean":"2024-09-13T17:45:26.881348+0000","last_became_active":"2024-09-13T17:45:26.873275+0000","last_became_peered":"2024-09-13T17:45:26.873275+0000","last_unstale":"2024-09-13T17:45:26.881348+0000","last_undegraded":"2024-09-13T17:45:26.881348+0000","last_fullsized":"2024-09-13T17:45:26.881348+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:03:05.368912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881329+0000","last_change":"2024-09-13T17:45:26.873635+0000","last_active":"2024-09-13T17:45:26.881329+0000","last_peered":"2024-09-13T17:45:26.881329+0000","last_clean":"2024-09-13T17:45:26.881329+0000","last_became_active":"2024-09-13T17:45:26.873374+0000","last_became_peered":"2024-09-13T17:45:26.873374+0000","last_unstale":"2024-09-13T17:45:26.881329+0000","last_undegraded":"2024-09-13T17:45:26.881329+0000","last_fullsized":"2024-09-13T17:45:26.881329+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:43:02.504758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884178+0000","last_change":"2024-09-13T17:45:26.877495+0000","last_active":"2024-09-13T17:45:26.884178+0000","last_peered":"2024-09-13T17:45:26.884178+0000","last_clean":"2024-09-13T17:45:26.884178+0000","last_became_active":"2024-09-13T17:45:26.877033+0000","last_became_peered":"2024-09-13T17:45:26.877033+0000","last_unstale":"2024-09-13T17:45:26.884178+0000","last_undegraded":"2024-09-13T17:45:26.884178+0000","last_fullsized":"2024-09-13T17:45:26.884178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:00:27.058685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859558+0000","last_change":"2024-09-13T17:45:26.855122+0000","last_active":"2024-09-13T17:45:26.859558+0000","last_peered":"2024-09-13T17:45:26.859558+0000","last_clean":"2024-09-13T17:45:26.859558+0000","last_became_active":"2024-09-13T17:45:26.854850+0000","last_became_peered":"2024-09-13T17:45:26.854850+0000","last_unstale":"2024-09-13T17:45:26.859558+0000","last_undegraded":"2024-09-13T17:45:26.859558+0000","last_fullsized":"2024-09-13T17:45:26.859558+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:07:11.062329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.870908+0000","last_change":"2024-09-13T17:45:26.867972+0000","last_active":"2024-09-13T17:45:26.870908+0000","last_peered":"2024-09-13T17:45:26.870908+0000","last_clean":"2024-09-13T17:45:26.870908+0000","last_became_active":"2024-09-13T17:45:26.867297+0000","last_became_peered":"2024-09-13T17:45:26.867297+0000","last_unstale":"2024-09-13T17:45:26.870908+0000","last_undegraded":"2024-09-13T17:45:26.870908+0000","last_fullsized":"2024-09-13T17:45:26.870908+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:35:13.541689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859198+0000","last_change":"2024-09-13T17:45:26.850945+0000","last_active":"2024-09-13T17:45:26.859198+0000","last_peered":"2024-09-13T17:45:26.859198+0000","last_clean":"2024-09-13T17:45:26.859198+0000","last_became_active":"2024-09-13T17:45:26.850261+0000","last_became_peered":"2024-09-13T17:45:26.850261+0000","last_unstale":"2024-09-13T17:45:26.859198+0000","last_undegraded":"2024-09-13T17:45:26.859198+0000","last_fullsized":"2024-09-13T17:45:26.859198+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:21.495952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881215+0000","last_change":"2024-09-13T17:45:26.875222+0000","last_active":"2024-09-13T17:45:26.881215+0000","last_peered":"2024-09-13T17:45:26.881215+0000","last_clean":"2024-09-13T17:45:26.881215+0000","last_became_active":"2024-09-13T17:45:26.874949+0000","last_became_peered":"2024-09-13T17:45:26.874949+0000","last_unstale":"2024-09-13T17:45:26.881215+0000","last_undegraded":"2024-09-13T17:45:26.881215+0000","last_fullsized":"2024-09-13T17:45:26.881215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:20.032657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865438+0000","last_change":"2024-09-13T17:45:26.861148+0000","last_active":"2024-09-13T17:45:26.865438+0000","last_peered":"2024-09-13T17:45:26.865438+0000","last_clean":"2024-09-13T17:45:26.865438+0000","last_became_active":"2024-09-13T17:45:26.860689+0000","last_became_peered":"2024-09-13T17:45:26.860689+0000","last_unstale":"2024-09-13T17:45:26.865438+0000","last_undegraded":"2024-09-13T17:45:26.865438+0000","last_fullsized":"2024-09-13T17:45:26.865438+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:16:29.009219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875556+0000","last_change":"2024-09-13T17:45:26.866583+0000","last_active":"2024-09-13T17:45:26.875556+0000","last_peered":"2024-09-13T17:45:26.875556+0000","last_clean":"2024-09-13T17:45:26.875556+0000","last_became_active":"2024-09-13T17:45:26.865997+0000","last_became_peered":"2024-09-13T17:45:26.865997+0000","last_unstale":"2024-09-13T17:45:26.875556+0000","last_undegraded":"2024-09-13T17:45:26.875556+0000","last_fullsized":"2024-09-13T17:45:26.875556+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:19:26.076527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"67'4","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875645+0000","last_change":"2024-09-13T17:45:26.470813+0000","last_active":"2024-09-13T17:45:26.875645+0000","last_peered":"2024-09-13T17:45:26.875645+0000","last_clean":"2024-09-13T17:45:26.875645+0000","last_became_active":"2024-09-13T17:45:26.470389+0000","last_became_peered":"2024-09-13T17:45:26.470389+0000","last_unstale":"2024-09-13T17:45:26.875645+0000","last_undegraded":"2024-09-13T17:45:26.875645+0000","last_fullsized":"2024-09-13T17:45:26.875645+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:09:14.941728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859601+0000","last_change":"2024-09-13T17:45:26.856407+0000","last_active":"2024-09-13T17:45:26.859601+0000","last_peered":"2024-09-13T17:45:26.859601+0000","last_clean":"2024-09-13T17:45:26.859601+0000","last_became_active":"2024-09-13T17:45:26.855265+0000","last_became_peered":"2024-09-13T17:45:26.855265+0000","last_unstale":"2024-09-13T17:45:26.859601+0000","last_undegraded":"2024-09-13T17:45:26.859601+0000","last_fullsized":"2024-09-13T17:45:26.859601+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:28:38.510250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.860242+0000","last_change":"2024-09-13T17:45:26.485330+0000","last_active":"2024-09-13T17:45:26.860242+0000","last_peered":"2024-09-13T17:45:26.860242+0000","last_clean":"2024-09-13T17:45:26.860242+0000","last_became_active":"2024-09-13T17:45:26.483742+0000","last_became_peered":"2024-09-13T17:45:26.483742+0000","last_unstale":"2024-09-13T17:45:26.860242+0000","last_undegraded":"2024-09-13T17:45:26.860242+0000","last_fullsized":"2024-09-13T17:45:26.860242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:09:36.396235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877222+0000","last_change":"2024-09-13T17:45:26.866163+0000","last_active":"2024-09-13T17:45:26.877222+0000","last_peered":"2024-09-13T17:45:26.877222+0000","last_clean":"2024-09-13T17:45:26.877222+0000","last_became_active":"2024-09-13T17:45:26.863234+0000","last_became_peered":"2024-09-13T17:45:26.863234+0000","last_unstale":"2024-09-13T17:45:26.877222+0000","last_undegraded":"2024-09-13T17:45:26.877222+0000","last_fullsized":"2024-09-13T17:45:26.877222+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:46:05.637403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881508+0000","last_change":"2024-09-13T17:45:26.476448+0000","last_active":"2024-09-13T17:45:26.881508+0000","last_peered":"2024-09-13T17:45:26.881508+0000","last_clean":"2024-09-13T17:45:26.881508+0000","last_became_active":"2024-09-13T17:45:26.476065+0000","last_became_peered":"2024-09-13T17:45:26.476065+0000","last_unstale":"2024-09-13T17:45:26.881508+0000","last_undegraded":"2024-09-13T17:45:26.881508+0000","last_fullsized":"2024-09-13T17:45:26.881508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:44:09.312668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859651+0000","last_change":"2024-09-13T17:45:26.855921+0000","last_active":"2024-09-13T17:45:26.859651+0000","last_peered":"2024-09-13T17:45:26.859651+0000","last_clean":"2024-09-13T17:45:26.859651+0000","last_became_active":"2024-09-13T17:45:26.855424+0000","last_became_peered":"2024-09-13T17:45:26.855424+0000","last_unstale":"2024-09-13T17:45:26.859651+0000","last_undegraded":"2024-09-13T17:45:26.859651+0000","last_fullsized":"2024-09-13T17:45:26.859651+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:33:50.343310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877325+0000","last_change":"2024-09-13T17:45:26.465677+0000","last_active":"2024-09-13T17:45:26.877325+0000","last_peered":"2024-09-13T17:45:26.877325+0000","last_clean":"2024-09-13T17:45:26.877325+0000","last_became_active":"2024-09-13T17:45:26.464625+0000","last_became_peered":"2024-09-13T17:45:26.464625+0000","last_unstale":"2024-09-13T17:45:26.877325+0000","last_undegraded":"2024-09-13T17:45:26.877325+0000","last_fullsized":"2024-09-13T17:45:26.877325+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:21:29.441415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859155+0000","last_change":"2024-09-13T17:45:26.850539+0000","last_active":"2024-09-13T17:45:26.859155+0000","last_peered":"2024-09-13T17:45:26.859155+0000","last_clean":"2024-09-13T17:45:26.859155+0000","last_became_active":"2024-09-13T17:45:26.849631+0000","last_became_peered":"2024-09-13T17:45:26.849631+0000","last_unstale":"2024-09-13T17:45:26.859155+0000","last_undegraded":"2024-09-13T17:45:26.859155+0000","last_fullsized":"2024-09-13T17:45:26.859155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:09:39.363449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875721+0000","last_change":"2024-09-13T17:45:26.466423+0000","last_active":"2024-09-13T17:45:26.875721+0000","last_peered":"2024-09-13T17:45:26.875721+0000","last_clean":"2024-09-13T17:45:26.875721+0000","last_became_active":"2024-09-13T17:45:26.465264+0000","last_became_peered":"2024-09-13T17:45:26.465264+0000","last_unstale":"2024-09-13T17:45:26.875721+0000","last_undegraded":"2024-09-13T17:45:26.875721+0000","last_fullsized":"2024-09-13T17:45:26.875721+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:17.357306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884238+0000","last_change":"2024-09-13T17:45:26.877910+0000","last_active":"2024-09-13T17:45:26.884238+0000","last_peered":"2024-09-13T17:45:26.884238+0000","last_clean":"2024-09-13T17:45:26.884238+0000","last_became_active":"2024-09-13T17:45:26.877022+0000","last_became_peered":"2024-09-13T17:45:26.877022+0000","last_unstale":"2024-09-13T17:45:26.884238+0000","last_undegraded":"2024-09-13T17:45:26.884238+0000","last_fullsized":"2024-09-13T17:45:26.884238+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:23:21.495952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.876598+0000","last_change":"2024-09-13T17:45:26.482402+0000","last_active":"2024-09-13T17:45:26.876598+0000","last_peered":"2024-09-13T17:45:26.876598+0000","last_clean":"2024-09-13T17:45:26.876598+0000","last_became_active":"2024-09-13T17:45:26.481148+0000","last_became_peered":"2024-09-13T17:45:26.481148+0000","last_unstale":"2024-09-13T17:45:26.876598+0000","last_undegraded":"2024-09-13T17:45:26.876598+0000","last_fullsized":"2024-09-13T17:45:26.876598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:16:25.050197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879505+0000","last_change":"2024-09-13T17:45:26.868558+0000","last_active":"2024-09-13T17:45:26.879505+0000","last_peered":"2024-09-13T17:45:26.879505+0000","last_clean":"2024-09-13T17:45:26.879505+0000","last_became_active":"2024-09-13T17:45:26.868051+0000","last_became_peered":"2024-09-13T17:45:26.868051+0000","last_unstale":"2024-09-13T17:45:26.879505+0000","last_undegraded":"2024-09-13T17:45:26.879505+0000","last_fullsized":"2024-09-13T17:45:26.879505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:56:59.853966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.847131+0000","last_change":"2024-09-13T17:45:26.462188+0000","last_active":"2024-09-13T17:45:26.847131+0000","last_peered":"2024-09-13T17:45:26.847131+0000","last_clean":"2024-09-13T17:45:26.847131+0000","last_became_active":"2024-09-13T17:45:26.461671+0000","last_became_peered":"2024-09-13T17:45:26.461671+0000","last_unstale":"2024-09-13T17:45:26.847131+0000","last_undegraded":"2024-09-13T17:45:26.847131+0000","last_fullsized":"2024-09-13T17:45:26.847131+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:13:00.822379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.865357+0000","last_change":"2024-09-13T17:45:26.860578+0000","last_active":"2024-09-13T17:45:26.865357+0000","last_peered":"2024-09-13T17:45:26.865357+0000","last_clean":"2024-09-13T17:45:26.865357+0000","last_became_active":"2024-09-13T17:45:26.860309+0000","last_became_peered":"2024-09-13T17:45:26.860309+0000","last_unstale":"2024-09-13T17:45:26.865357+0000","last_undegraded":"2024-09-13T17:45:26.865357+0000","last_fullsized":"2024-09-13T17:45:26.865357+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:00:57.480806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.838045+0000","last_change":"2024-09-13T17:45:26.473990+0000","last_active":"2024-09-13T17:45:26.838045+0000","last_peered":"2024-09-13T17:45:26.838045+0000","last_clean":"2024-09-13T17:45:26.838045+0000","last_became_active":"2024-09-13T17:45:26.468331+0000","last_became_peered":"2024-09-13T17:45:26.468331+0000","last_unstale":"2024-09-13T17:45:26.838045+0000","last_undegraded":"2024-09-13T17:45:26.838045+0000","last_fullsized":"2024-09-13T17:45:26.838045+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:02:43.592965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875500+0000","last_change":"2024-09-13T17:45:26.866392+0000","last_active":"2024-09-13T17:45:26.875500+0000","last_peered":"2024-09-13T17:45:26.875500+0000","last_clean":"2024-09-13T17:45:26.875500+0000","last_became_active":"2024-09-13T17:45:26.862928+0000","last_became_peered":"2024-09-13T17:45:26.862928+0000","last_unstale":"2024-09-13T17:45:26.875500+0000","last_undegraded":"2024-09-13T17:45:26.875500+0000","last_fullsized":"2024-09-13T17:45:26.875500+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:29:30.421111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842429+0000","last_change":"2024-09-13T17:45:26.476448+0000","last_active":"2024-09-13T17:45:26.842429+0000","last_peered":"2024-09-13T17:45:26.842429+0000","last_clean":"2024-09-13T17:45:26.842429+0000","last_became_active":"2024-09-13T17:45:26.475617+0000","last_became_peered":"2024-09-13T17:45:26.475617+0000","last_unstale":"2024-09-13T17:45:26.842429+0000","last_undegraded":"2024-09-13T17:45:26.842429+0000","last_fullsized":"2024-09-13T17:45:26.842429+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:19:22.785469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859583+0000","last_change":"2024-09-13T17:45:26.856239+0000","last_active":"2024-09-13T17:45:26.859583+0000","last_peered":"2024-09-13T17:45:26.859583+0000","last_clean":"2024-09-13T17:45:26.859583+0000","last_became_active":"2024-09-13T17:45:26.855030+0000","last_became_peered":"2024-09-13T17:45:26.855030+0000","last_unstale":"2024-09-13T17:45:26.859583+0000","last_undegraded":"2024-09-13T17:45:26.859583+0000","last_fullsized":"2024-09-13T17:45:26.859583+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:50:42.690789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.850967+0000","last_change":"2024-09-13T17:45:26.482572+0000","last_active":"2024-09-13T17:45:26.850967+0000","last_peered":"2024-09-13T17:45:26.850967+0000","last_clean":"2024-09-13T17:45:26.850967+0000","last_became_active":"2024-09-13T17:45:26.481852+0000","last_became_peered":"2024-09-13T17:45:26.481852+0000","last_unstale":"2024-09-13T17:45:26.850967+0000","last_undegraded":"2024-09-13T17:45:26.850967+0000","last_fullsized":"2024-09-13T17:45:26.850967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:58:04.925857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877201+0000","last_change":"2024-09-13T17:45:26.865172+0000","last_active":"2024-09-13T17:45:26.877201+0000","last_peered":"2024-09-13T17:45:26.877201+0000","last_clean":"2024-09-13T17:45:26.877201+0000","last_became_active":"2024-09-13T17:45:26.862931+0000","last_became_peered":"2024-09-13T17:45:26.862931+0000","last_unstale":"2024-09-13T17:45:26.877201+0000","last_undegraded":"2024-09-13T17:45:26.877201+0000","last_fullsized":"2024-09-13T17:45:26.877201+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:33:33.221904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"67'3","reported_seq":13,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859520+0000","last_change":"2024-09-13T17:45:26.475217+0000","last_active":"2024-09-13T17:45:26.859520+0000","last_peered":"2024-09-13T17:45:26.859520+0000","last_clean":"2024-09-13T17:45:26.859520+0000","last_became_active":"2024-09-13T17:45:26.474944+0000","last_became_peered":"2024-09-13T17:45:26.474944+0000","last_unstale":"2024-09-13T17:45:26.859520+0000","last_undegraded":"2024-09-13T17:45:26.859520+0000","last_fullsized":"2024-09-13T17:45:26.859520+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:24:32.426871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884199+0000","last_change":"2024-09-13T17:45:26.877638+0000","last_active":"2024-09-13T17:45:26.884199+0000","last_peered":"2024-09-13T17:45:26.884199+0000","last_clean":"2024-09-13T17:45:26.884199+0000","last_became_active":"2024-09-13T17:45:26.877062+0000","last_became_peered":"2024-09-13T17:45:26.877062+0000","last_unstale":"2024-09-13T17:45:26.884199+0000","last_undegraded":"2024-09-13T17:45:26.884199+0000","last_fullsized":"2024-09-13T17:45:26.884199+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:05:43.965608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.14","version":"67'6","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.886860+0000","last_change":"2024-09-13T17:45:26.466199+0000","last_active":"2024-09-13T17:45:26.886860+0000","last_peered":"2024-09-13T17:45:26.886860+0000","last_clean":"2024-09-13T17:45:26.886860+0000","last_became_active":"2024-09-13T17:45:26.464106+0000","last_became_peered":"2024-09-13T17:45:26.464106+0000","last_unstale":"2024-09-13T17:45:26.886860+0000","last_undegraded":"2024-09-13T17:45:26.886860+0000","last_fullsized":"2024-09-13T17:45:26.886860+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:04:50.667743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.874998+0000","last_change":"2024-09-13T17:45:26.862455+0000","last_active":"2024-09-13T17:45:26.874998+0000","last_peered":"2024-09-13T17:45:26.874998+0000","last_clean":"2024-09-13T17:45:26.874998+0000","last_became_active":"2024-09-13T17:45:26.861182+0000","last_became_peered":"2024-09-13T17:45:26.861182+0000","last_unstale":"2024-09-13T17:45:26.874998+0000","last_undegraded":"2024-09-13T17:45:26.874998+0000","last_fullsized":"2024-09-13T17:45:26.874998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:13:35.493081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877761+0000","last_change":"2024-09-13T17:45:26.468317+0000","last_active":"2024-09-13T17:45:26.877761+0000","last_peered":"2024-09-13T17:45:26.877761+0000","last_clean":"2024-09-13T17:45:26.877761+0000","last_became_active":"2024-09-13T17:45:26.460439+0000","last_became_peered":"2024-09-13T17:45:26.460439+0000","last_unstale":"2024-09-13T17:45:26.877761+0000","last_undegraded":"2024-09-13T17:45:26.877761+0000","last_fullsized":"2024-09-13T17:45:26.877761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:57.445897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881430+0000","last_change":"2024-09-13T17:45:26.878107+0000","last_active":"2024-09-13T17:45:26.881430+0000","last_peered":"2024-09-13T17:45:26.881430+0000","last_clean":"2024-09-13T17:45:26.881430+0000","last_became_active":"2024-09-13T17:45:26.877844+0000","last_became_peered":"2024-09-13T17:45:26.877844+0000","last_unstale":"2024-09-13T17:45:26.881430+0000","last_undegraded":"2024-09-13T17:45:26.881430+0000","last_fullsized":"2024-09-13T17:45:26.881430+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:54:45.537512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846173+0000","last_change":"2024-09-13T17:45:26.466173+0000","last_active":"2024-09-13T17:45:26.846173+0000","last_peered":"2024-09-13T17:45:26.846173+0000","last_clean":"2024-09-13T17:45:26.846173+0000","last_became_active":"2024-09-13T17:45:26.464683+0000","last_became_peered":"2024-09-13T17:45:26.464683+0000","last_unstale":"2024-09-13T17:45:26.846173+0000","last_undegraded":"2024-09-13T17:45:26.846173+0000","last_fullsized":"2024-09-13T17:45:26.846173+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:17:23.116548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859013+0000","last_change":"2024-09-13T17:45:26.853330+0000","last_active":"2024-09-13T17:45:26.859013+0000","last_peered":"2024-09-13T17:45:26.859013+0000","last_clean":"2024-09-13T17:45:26.859013+0000","last_became_active":"2024-09-13T17:45:26.852784+0000","last_became_peered":"2024-09-13T17:45:26.852784+0000","last_unstale":"2024-09-13T17:45:26.859013+0000","last_undegraded":"2024-09-13T17:45:26.859013+0000","last_fullsized":"2024-09-13T17:45:26.859013+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:02.152157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842223+0000","last_change":"2024-09-13T17:45:26.485776+0000","last_active":"2024-09-13T17:45:26.842223+0000","last_peered":"2024-09-13T17:45:26.842223+0000","last_clean":"2024-09-13T17:45:26.842223+0000","last_became_active":"2024-09-13T17:45:26.485473+0000","last_became_peered":"2024-09-13T17:45:26.485473+0000","last_unstale":"2024-09-13T17:45:26.842223+0000","last_undegraded":"2024-09-13T17:45:26.842223+0000","last_fullsized":"2024-09-13T17:45:26.842223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:23:52.070614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879155+0000","last_change":"2024-09-13T17:45:26.870500+0000","last_active":"2024-09-13T17:45:26.879155+0000","last_peered":"2024-09-13T17:45:26.879155+0000","last_clean":"2024-09-13T17:45:26.879155+0000","last_became_active":"2024-09-13T17:45:26.870245+0000","last_became_peered":"2024-09-13T17:45:26.870245+0000","last_unstale":"2024-09-13T17:45:26.879155+0000","last_undegraded":"2024-09-13T17:45:26.879155+0000","last_fullsized":"2024-09-13T17:45:26.879155+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:32:40.137656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"67'8","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.879874+0000","last_change":"2024-09-13T17:45:26.464589+0000","last_active":"2024-09-13T17:45:26.879874+0000","last_peered":"2024-09-13T17:45:26.879874+0000","last_clean":"2024-09-13T17:45:26.879874+0000","last_became_active":"2024-09-13T17:45:26.463835+0000","last_became_peered":"2024-09-13T17:45:26.463835+0000","last_unstale":"2024-09-13T17:45:26.879874+0000","last_undegraded":"2024-09-13T17:45:26.879874+0000","last_fullsized":"2024-09-13T17:45:26.879874+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:55:18.186378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875581+0000","last_change":"2024-09-13T17:45:26.868705+0000","last_active":"2024-09-13T17:45:26.875581+0000","last_peered":"2024-09-13T17:45:26.875581+0000","last_clean":"2024-09-13T17:45:26.875581+0000","last_became_active":"2024-09-13T17:45:26.866216+0000","last_became_peered":"2024-09-13T17:45:26.866216+0000","last_unstale":"2024-09-13T17:45:26.875581+0000","last_undegraded":"2024-09-13T17:45:26.875581+0000","last_fullsized":"2024-09-13T17:45:26.875581+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:48:10.889384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872859+0000","last_change":"2024-09-13T17:45:26.470652+0000","last_active":"2024-09-13T17:45:26.872859+0000","last_peered":"2024-09-13T17:45:26.872859+0000","last_clean":"2024-09-13T17:45:26.872859+0000","last_became_active":"2024-09-13T17:45:26.470068+0000","last_became_peered":"2024-09-13T17:45:26.470068+0000","last_unstale":"2024-09-13T17:45:26.872859+0000","last_undegraded":"2024-09-13T17:45:26.872859+0000","last_fullsized":"2024-09-13T17:45:26.872859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:31:02.385141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884083+0000","last_change":"2024-09-13T17:45:26.876751+0000","last_active":"2024-09-13T17:45:26.884083+0000","last_peered":"2024-09-13T17:45:26.884083+0000","last_clean":"2024-09-13T17:45:26.884083+0000","last_became_active":"2024-09-13T17:45:26.876443+0000","last_became_peered":"2024-09-13T17:45:26.876443+0000","last_unstale":"2024-09-13T17:45:26.884083+0000","last_undegraded":"2024-09-13T17:45:26.884083+0000","last_fullsized":"2024-09-13T17:45:26.884083+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:52:18.186744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.872013+0000","last_change":"2024-09-13T17:45:26.465582+0000","last_active":"2024-09-13T17:45:26.872013+0000","last_peered":"2024-09-13T17:45:26.872013+0000","last_clean":"2024-09-13T17:45:26.872013+0000","last_became_active":"2024-09-13T17:45:26.465082+0000","last_became_peered":"2024-09-13T17:45:26.465082+0000","last_unstale":"2024-09-13T17:45:26.872013+0000","last_undegraded":"2024-09-13T17:45:26.872013+0000","last_fullsized":"2024-09-13T17:45:26.872013+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:38:53.048020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875418+0000","last_change":"2024-09-13T17:45:26.864827+0000","last_active":"2024-09-13T17:45:26.875418+0000","last_peered":"2024-09-13T17:45:26.875418+0000","last_clean":"2024-09-13T17:45:26.875418+0000","last_became_active":"2024-09-13T17:45:26.861456+0000","last_became_peered":"2024-09-13T17:45:26.861456+0000","last_unstale":"2024-09-13T17:45:26.875418+0000","last_undegraded":"2024-09-13T17:45:26.875418+0000","last_fullsized":"2024-09-13T17:45:26.875418+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T01:44:19.479096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846501+0000","last_change":"2024-09-13T17:45:26.466500+0000","last_active":"2024-09-13T17:45:26.846501+0000","last_peered":"2024-09-13T17:45:26.846501+0000","last_clean":"2024-09-13T17:45:26.846501+0000","last_became_active":"2024-09-13T17:45:26.465685+0000","last_became_peered":"2024-09-13T17:45:26.465685+0000","last_unstale":"2024-09-13T17:45:26.846501+0000","last_undegraded":"2024-09-13T17:45:26.846501+0000","last_fullsized":"2024-09-13T17:45:26.846501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:32:28.225692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.863832+0000","last_change":"2024-09-13T17:45:26.858987+0000","last_active":"2024-09-13T17:45:26.863832+0000","last_peered":"2024-09-13T17:45:26.863832+0000","last_clean":"2024-09-13T17:45:26.863832+0000","last_became_active":"2024-09-13T17:45:26.858742+0000","last_became_peered":"2024-09-13T17:45:26.858742+0000","last_unstale":"2024-09-13T17:45:26.863832+0000","last_undegraded":"2024-09-13T17:45:26.863832+0000","last_fullsized":"2024-09-13T17:45:26.863832+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:00:06.676044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875791+0000","last_change":"2024-09-13T17:45:26.471744+0000","last_active":"2024-09-13T17:45:26.875791+0000","last_peered":"2024-09-13T17:45:26.875791+0000","last_clean":"2024-09-13T17:45:26.875791+0000","last_became_active":"2024-09-13T17:45:26.471366+0000","last_became_peered":"2024-09-13T17:45:26.471366+0000","last_unstale":"2024-09-13T17:45:26.875791+0000","last_undegraded":"2024-09-13T17:45:26.875791+0000","last_fullsized":"2024-09-13T17:45:26.875791+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:52:47.021350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884226+0000","last_change":"2024-09-13T17:45:26.877656+0000","last_active":"2024-09-13T17:45:26.884226+0000","last_peered":"2024-09-13T17:45:26.884226+0000","last_clean":"2024-09-13T17:45:26.884226+0000","last_became_active":"2024-09-13T17:45:26.877306+0000","last_became_peered":"2024-09-13T17:45:26.877306+0000","last_unstale":"2024-09-13T17:45:26.884226+0000","last_undegraded":"2024-09-13T17:45:26.884226+0000","last_fullsized":"2024-09-13T17:45:26.884226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:15.557735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841766+0000","last_change":"2024-09-13T17:45:26.482746+0000","last_active":"2024-09-13T17:45:26.841766+0000","last_peered":"2024-09-13T17:45:26.841766+0000","last_clean":"2024-09-13T17:45:26.841766+0000","last_became_active":"2024-09-13T17:45:26.482078+0000","last_became_peered":"2024-09-13T17:45:26.482078+0000","last_unstale":"2024-09-13T17:45:26.841766+0000","last_undegraded":"2024-09-13T17:45:26.841766+0000","last_fullsized":"2024-09-13T17:45:26.841766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:51:45.524332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881298+0000","last_change":"2024-09-13T17:45:26.875733+0000","last_active":"2024-09-13T17:45:26.881298+0000","last_peered":"2024-09-13T17:45:26.881298+0000","last_clean":"2024-09-13T17:45:26.881298+0000","last_became_active":"2024-09-13T17:45:26.875464+0000","last_became_peered":"2024-09-13T17:45:26.875464+0000","last_unstale":"2024-09-13T17:45:26.881298+0000","last_undegraded":"2024-09-13T17:45:26.881298+0000","last_fullsized":"2024-09-13T17:45:26.881298+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:41:02.152157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.842326+0000","last_change":"2024-09-13T17:45:26.485613+0000","last_active":"2024-09-13T17:45:26.842326+0000","last_peered":"2024-09-13T17:45:26.842326+0000","last_clean":"2024-09-13T17:45:26.842326+0000","last_became_active":"2024-09-13T17:45:26.485333+0000","last_became_peered":"2024-09-13T17:45:26.485333+0000","last_unstale":"2024-09-13T17:45:26.842326+0000","last_undegraded":"2024-09-13T17:45:26.842326+0000","last_fullsized":"2024-09-13T17:45:26.842326+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:48:16.875343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875471+0000","last_change":"2024-09-13T17:45:26.866094+0000","last_active":"2024-09-13T17:45:26.875471+0000","last_peered":"2024-09-13T17:45:26.875471+0000","last_clean":"2024-09-13T17:45:26.875471+0000","last_became_active":"2024-09-13T17:45:26.862870+0000","last_became_peered":"2024-09-13T17:45:26.862870+0000","last_unstale":"2024-09-13T17:45:26.875471+0000","last_undegraded":"2024-09-13T17:45:26.875471+0000","last_fullsized":"2024-09-13T17:45:26.875471+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:32:12.114032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841265+0000","last_change":"2024-09-13T17:45:26.481550+0000","last_active":"2024-09-13T17:45:26.841265+0000","last_peered":"2024-09-13T17:45:26.841265+0000","last_clean":"2024-09-13T17:45:26.841265+0000","last_became_active":"2024-09-13T17:45:26.481128+0000","last_became_peered":"2024-09-13T17:45:26.481128+0000","last_unstale":"2024-09-13T17:45:26.841265+0000","last_undegraded":"2024-09-13T17:45:26.841265+0000","last_fullsized":"2024-09-13T17:45:26.841265+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T22:24:59.217212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.874938+0000","last_change":"2024-09-13T17:45:26.863363+0000","last_active":"2024-09-13T17:45:26.874938+0000","last_peered":"2024-09-13T17:45:26.874938+0000","last_clean":"2024-09-13T17:45:26.874938+0000","last_became_active":"2024-09-13T17:45:26.862490+0000","last_became_peered":"2024-09-13T17:45:26.862490+0000","last_unstale":"2024-09-13T17:45:26.874938+0000","last_undegraded":"2024-09-13T17:45:26.874938+0000","last_fullsized":"2024-09-13T17:45:26.874938+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T00:12:06.864302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.7","version":"67'2","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.858402+0000","last_change":"2024-09-13T17:45:26.476236+0000","last_active":"2024-09-13T17:45:26.858402+0000","last_peered":"2024-09-13T17:45:26.858402+0000","last_clean":"2024-09-13T17:45:26.858402+0000","last_became_active":"2024-09-13T17:45:26.474740+0000","last_became_peered":"2024-09-13T17:45:26.474740+0000","last_unstale":"2024-09-13T17:45:26.858402+0000","last_undegraded":"2024-09-13T17:45:26.858402+0000","last_fullsized":"2024-09-13T17:45:26.858402+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:34:05.270907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.881373+0000","last_change":"2024-09-13T17:45:26.873825+0000","last_active":"2024-09-13T17:45:26.881373+0000","last_peered":"2024-09-13T17:45:26.881373+0000","last_clean":"2024-09-13T17:45:26.881373+0000","last_became_active":"2024-09-13T17:45:26.873383+0000","last_became_peered":"2024-09-13T17:45:26.873383+0000","last_unstale":"2024-09-13T17:45:26.881373+0000","last_undegraded":"2024-09-13T17:45:26.881373+0000","last_fullsized":"2024-09-13T17:45:26.881373+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:46:33.687155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841901+0000","last_change":"2024-09-13T17:45:26.468248+0000","last_active":"2024-09-13T17:45:26.841901+0000","last_peered":"2024-09-13T17:45:26.841901+0000","last_clean":"2024-09-13T17:45:26.841901+0000","last_became_active":"2024-09-13T17:45:26.466994+0000","last_became_peered":"2024-09-13T17:45:26.466994+0000","last_unstale":"2024-09-13T17:45:26.841901+0000","last_undegraded":"2024-09-13T17:45:26.841901+0000","last_fullsized":"2024-09-13T17:45:26.841901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:26:42.730586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.877262+0000","last_change":"2024-09-13T17:45:26.866560+0000","last_active":"2024-09-13T17:45:26.877262+0000","last_peered":"2024-09-13T17:45:26.877262+0000","last_clean":"2024-09-13T17:45:26.877262+0000","last_became_active":"2024-09-13T17:45:26.864707+0000","last_became_peered":"2024-09-13T17:45:26.864707+0000","last_unstale":"2024-09-13T17:45:26.877262+0000","last_undegraded":"2024-09-13T17:45:26.877262+0000","last_fullsized":"2024-09-13T17:45:26.877262+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:37:35.606328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846766+0000","last_change":"2024-09-13T17:45:26.475689+0000","last_active":"2024-09-13T17:45:26.846766+0000","last_peered":"2024-09-13T17:45:26.846766+0000","last_clean":"2024-09-13T17:45:26.846766+0000","last_became_active":"2024-09-13T17:45:26.470568+0000","last_became_peered":"2024-09-13T17:45:26.470568+0000","last_unstale":"2024-09-13T17:45:26.846766+0000","last_undegraded":"2024-09-13T17:45:26.846766+0000","last_fullsized":"2024-09-13T17:45:26.846766+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:19:51.371173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.884157+0000","last_change":"2024-09-13T17:45:26.877467+0000","last_active":"2024-09-13T17:45:26.884157+0000","last_peered":"2024-09-13T17:45:26.884157+0000","last_clean":"2024-09-13T17:45:26.884157+0000","last_became_active":"2024-09-13T17:45:26.876694+0000","last_became_peered":"2024-09-13T17:45:26.876694+0000","last_unstale":"2024-09-13T17:45:26.884157+0000","last_undegraded":"2024-09-13T17:45:26.884157+0000","last_fullsized":"2024-09-13T17:45:26.884157+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:44:09.564935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.847759+0000","last_change":"2024-09-13T17:45:26.483859+0000","last_active":"2024-09-13T17:45:26.847759+0000","last_peered":"2024-09-13T17:45:26.847759+0000","last_clean":"2024-09-13T17:45:26.847759+0000","last_became_active":"2024-09-13T17:45:26.483529+0000","last_became_peered":"2024-09-13T17:45:26.483529+0000","last_unstale":"2024-09-13T17:45:26.847759+0000","last_undegraded":"2024-09-13T17:45:26.847759+0000","last_fullsized":"2024-09-13T17:45:26.847759+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T18:29:07.902010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859178+0000","last_change":"2024-09-13T17:45:26.850728+0000","last_active":"2024-09-13T17:45:26.859178+0000","last_peered":"2024-09-13T17:45:26.859178+0000","last_clean":"2024-09-13T17:45:26.859178+0000","last_became_active":"2024-09-13T17:45:26.849950+0000","last_became_peered":"2024-09-13T17:45:26.849950+0000","last_unstale":"2024-09-13T17:45:26.859178+0000","last_undegraded":"2024-09-13T17:45:26.859178+0000","last_fullsized":"2024-09-13T17:45:26.859178+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T05:27:37.261491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.846859+0000","last_change":"2024-09-13T17:45:26.471586+0000","last_active":"2024-09-13T17:45:26.846859+0000","last_peered":"2024-09-13T17:45:26.846859+0000","last_clean":"2024-09-13T17:45:26.846859+0000","last_became_active":"2024-09-13T17:45:26.471071+0000","last_became_peered":"2024-09-13T17:45:26.471071+0000","last_unstale":"2024-09-13T17:45:26.846859+0000","last_undegraded":"2024-09-13T17:45:26.846859+0000","last_fullsized":"2024-09-13T17:45:26.846859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:48:51.549791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.875593+0000","last_change":"2024-09-13T17:45:26.864664+0000","last_active":"2024-09-13T17:45:26.875593+0000","last_peered":"2024-09-13T17:45:26.875593+0000","last_clean":"2024-09-13T17:45:26.875593+0000","last_became_active":"2024-09-13T17:45:26.862643+0000","last_became_peered":"2024-09-13T17:45:26.862643+0000","last_unstale":"2024-09-13T17:45:26.875593+0000","last_undegraded":"2024-09-13T17:45:26.875593+0000","last_fullsized":"2024-09-13T17:45:26.875593+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:19:14.923370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.841818+0000","last_change":"2024-09-13T17:45:26.482889+0000","last_active":"2024-09-13T17:45:26.841818+0000","last_peered":"2024-09-13T17:45:26.841818+0000","last_clean":"2024-09-13T17:45:26.841818+0000","last_became_active":"2024-09-13T17:45:26.482143+0000","last_became_peered":"2024-09-13T17:45:26.482143+0000","last_unstale":"2024-09-13T17:45:26.841818+0000","last_undegraded":"2024-09-13T17:45:26.841818+0000","last_fullsized":"2024-09-13T17:45:26.841818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T03:42:18.594276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859032+0000","last_change":"2024-09-13T17:45:26.853408+0000","last_active":"2024-09-13T17:45:26.859032+0000","last_peered":"2024-09-13T17:45:26.859032+0000","last_clean":"2024-09-13T17:45:26.859032+0000","last_became_active":"2024-09-13T17:45:26.853053+0000","last_became_peered":"2024-09-13T17:45:26.853053+0000","last_unstale":"2024-09-13T17:45:26.859032+0000","last_undegraded":"2024-09-13T17:45:26.859032+0000","last_fullsized":"2024-09-13T17:45:26.859032+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T04:57:19.666501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.837493+0000","last_change":"2024-09-13T17:45:26.460266+0000","last_active":"2024-09-13T17:45:26.837493+0000","last_peered":"2024-09-13T17:45:26.837493+0000","last_clean":"2024-09-13T17:45:26.837493+0000","last_became_active":"2024-09-13T17:45:26.459918+0000","last_became_peered":"2024-09-13T17:45:26.459918+0000","last_unstale":"2024-09-13T17:45:26.837493+0000","last_undegraded":"2024-09-13T17:45:26.837493+0000","last_fullsized":"2024-09-13T17:45:26.837493+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:13:43.446072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.858810+0000","last_change":"2024-09-13T17:45:26.853333+0000","last_active":"2024-09-13T17:45:26.858810+0000","last_peered":"2024-09-13T17:45:26.858810+0000","last_clean":"2024-09-13T17:45:26.858810+0000","last_became_active":"2024-09-13T17:45:26.853073+0000","last_became_peered":"2024-09-13T17:45:26.853073+0000","last_unstale":"2024-09-13T17:45:26.858810+0000","last_undegraded":"2024-09-13T17:45:26.858810+0000","last_fullsized":"2024-09-13T17:45:26.858810+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:51:15.557735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.848208+0000","last_change":"2024-09-13T17:45:26.465422+0000","last_active":"2024-09-13T17:45:26.848208+0000","last_peered":"2024-09-13T17:45:26.848208+0000","last_clean":"2024-09-13T17:45:26.848208+0000","last_became_active":"2024-09-13T17:45:26.464899+0000","last_became_peered":"2024-09-13T17:45:26.464899+0000","last_unstale":"2024-09-13T17:45:26.848208+0000","last_undegraded":"2024-09-13T17:45:26.848208+0000","last_fullsized":"2024-09-13T17:45:26.848208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T19:44:04.308482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.868211+0000","last_change":"2024-09-13T17:45:26.867663+0000","last_active":"2024-09-13T17:45:26.868211+0000","last_peered":"2024-09-13T17:45:26.868211+0000","last_clean":"2024-09-13T17:45:26.868211+0000","last_became_active":"2024-09-13T17:45:26.867406+0000","last_became_peered":"2024-09-13T17:45:26.867406+0000","last_unstale":"2024-09-13T17:45:26.868211+0000","last_undegraded":"2024-09-13T17:45:26.868211+0000","last_fullsized":"2024-09-13T17:45:26.868211+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:26.426121+0000","last_clean_scrub_stamp":"2024-09-13T17:45:26.426121+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T21:29:07.999777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"67'1","reported_seq":11,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:26.859438+0000","last_change":"2024-09-13T17:45:26.475549+0000","last_active":"2024-09-13T17:45:26.859438+0000","last_peered":"2024-09-13T17:45:26.859438+0000","last_clean":"2024-09-13T17:45:26.859438+0000","last_became_active":"2024-09-13T17:45:26.475160+0000","last_became_peered":"2024-09-13T17:45:26.475160+0000","last_unstale":"2024-09-13T17:45:26.859438+0000","last_undegraded":"2024-09-13T17:45:26.859438+0000","last_fullsized":"2024-09-13T17:45:26.859438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:45:25.416354+0000","last_clean_scrub_stamp":"2024-09-13T17:45:25.416354+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T20:42:32.768874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"67'1925","reported_seq":2162,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-13T17:45:27.533172+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:27.533172+0000","last_peered":"2024-09-13T17:45:27.533172+0000","last_clean":"2024-09-13T17:45:27.533172+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:27.533172+0000","last_undegraded":"2024-09-13T17:45:27.533172+0000","last_fullsized":"2024-09-13T17:45:27.533172+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":1925,"log_dups_size":0,"ondisk_log_size":1925,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":223,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":552,"num_read_kb":1033,"num_write":223,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1925,"ondisk_log_size":1925,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738413,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31780,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340060,"statfs":{"total":96636764160,"available":96604221440,"internally_reserved":0,"allocated":671744,"data_stored":256771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.21299999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.27200000000000002}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.38300000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738414,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36416,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335424,"statfs":{"total":96636764160,"available":96599474176,"internally_reserved":0,"allocated":1228800,"data_stored":840346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.378}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.41299999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738413,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40548,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331292,"statfs":{"total":96636764160,"available":96595243008,"internally_reserved":0,"allocated":1261568,"data_stored":847013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:31.269 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-09-13T17:45:31.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-13T17:45:31.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6d012640 1 Processor -- start 2024-09-13T17:45:31.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6d012640 1 -- start start 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6d012640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f68071500 0x7f8f6813adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6d012640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f68071b10 con 0x7f8f6812f530 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6d012640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f68070550 con 0x7f8f68071500 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f6659c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8f68071500 0x7f8f6813adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f66d9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8f6812f530 0x7f8f6812f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52316/0 (socket says 172.21.15.96:52316) 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.446+0000 7f8f66d9d640 1 -- 172.21.15.96:0/1311123650 learned_addr learned my addr 172.21.15.96:0/1311123650 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6659c640 1 -- 172.21.15.96:0/1311123650 >> v1:172.21.15.96:6789/0 conn(0x7f8f6812f530 legacy=0x7f8f6812f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6659c640 1 -- 172.21.15.96:0/1311123650 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6806fc40 con 0x7f8f68071500 2024-09-13T17:45:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6659c640 1 --2- 172.21.15.96:0/1311123650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813adb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009740 tx=0x7f8f5c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db5b32f71397722 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f65d9b640 1 -- 172.21.15.96:0/1311123650 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c0ab530 con 0x7f8f68071500 2024-09-13T17:45:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f65d9b640 1 -- 172.21.15.96:0/1311123650 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c0a7050 con 0x7f8f68071500 2024-09-13T17:45:31.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f65d9b640 1 -- 172.21.15.96:0/1311123650 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f5c0ab7e0 con 0x7f8f68071500 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 -- 172.21.15.96:0/1311123650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 msgr2=0x7f8f6813adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1311123650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813adb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009740 tx=0x7f8f5c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 -- 172.21.15.96:0/1311123650 shutdown_connections 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1311123650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813adb0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 -- 172.21.15.96:0/1311123650 >> 172.21.15.96:0/1311123650 conn(0x7f8f6806a150 msgr2=0x7f8f6806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 -- 172.21.15.96:0/1311123650 shutdown_connections 2024-09-13T17:45:31.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.450+0000 7f8f6d012640 1 -- 172.21.15.96:0/1311123650 wait complete. 2024-09-13T17:45:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 Processor -- start 2024-09-13T17:45:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 -- start start 2024-09-13T17:45:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6809c100 con 0x7f8f68071500 2024-09-13T17:45:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f66d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46006/0 (socket says 172.21.15.96:46006) 2024-09-13T17:45:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f66d9d640 1 -- 172.21.15.96:0/1517284656 learned_addr learned my addr 172.21.15.96:0/1517284656 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f66d9d640 1 -- 172.21.15.96:0/1517284656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c009420 con 0x7f8f68071500 2024-09-13T17:45:31.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f66d9d640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c480 tx=0x7f8f5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f58005c20 con 0x7f8f68071500 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f58005d80 con 0x7f8f68071500 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6809bc70 con 0x7f8f68071500 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f580071a0 con 0x7f8f68071500 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f681332f0 con 0x7f8f68071500 2024-09-13T17:45:31.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f8f58007300 con 0x7f8f68071500 2024-09-13T17:45:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f680a2790 con 0x7f8f68071500 2024-09-13T17:45:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 0x7f8f540379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f8f58046ac0 con 0x7f8f68071500 2024-09-13T17:45:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6659c640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 0x7f8f540379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.454+0000 7f8f6659c640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 0x7f8f540379a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0ad000 tx=0x7f8f5c008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.462+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8f680a2790 con 0x7f8f68071500 2024-09-13T17:45:31.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.690+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8f68071b10 con 0x7f8f68071500 2024-09-13T17:45:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f4f7fe640 1 -- 172.21.15.96:0/1517284656 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f8f68071b10 con 0x7f8f68071500 2024-09-13T17:45:31.696 INFO:teuthology.orchestra.run.smithi096.stderr:updated caps for client.0 2024-09-13T17:45:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 msgr2=0x7f8f540379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 0x7f8f540379a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0ad000 tx=0x7f8f5c008040 comp rx=0 tx=0).stop 2024-09-13T17:45:31.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 msgr2=0x7f8f6813a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c480 tx=0x7f8f5800c940 comp rx=0 tx=0).stop 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 shutdown_connections 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f8f540354e0 0x7f8f540379a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 --2- 172.21.15.96:0/1517284656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68071500 0x7f8f6813a0e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 >> 172.21.15.96:0/1517284656 conn(0x7f8f6806a150 msgr2=0x7f8f68130810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 shutdown_connections 2024-09-13T17:45:31.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.694+0000 7f8f6d012640 1 -- 172.21.15.96:0/1517284656 wait complete. 2024-09-13T17:45:31.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f24a5f13640 1 Processor -- start 2024-09-13T17:45:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f24a5f13640 1 -- start start 2024-09-13T17:45:31.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f24a5f13640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f24a00a4900 0x7f24a00a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f24a5f13640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24a0070600 con 0x7f24a00a6510 2024-09-13T17:45:31.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f24a5f13640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a0070830 con 0x7f24a00a4900 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f24a00a4900 0x7f24a00a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f24a00a6510 0x7f24a012f170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:52332/0 (socket says 172.21.15.96:52332) 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249f7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f24a00a4900 0x7f24a00a4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46016/0 (socket says 172.21.15.96:46016) 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249effd640 1 -- 172.21.15.96:0/236499683 learned_addr learned my addr 172.21.15.96:0/236499683 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249f7fe640 1 -- 172.21.15.96:0/236499683 >> v1:172.21.15.96:6789/0 conn(0x7f24a00a6510 legacy=0x7f24a012f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249f7fe640 1 -- 172.21.15.96:0/236499683 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a00a5c70 con 0x7f24a00a4900 2024-09-13T17:45:31.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249e7fc640 1 -- 172.21.15.96:0/236499683 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532272594 0 0) 0x7f24a0070600 con 0x7f24a00a6510 2024-09-13T17:45:31.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249f7fe640 1 --2- 172.21.15.96:0/236499683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a4900 0x7f24a00a4ce0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2490009740 tx=0x7f249001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c541c887b1f6c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.838+0000 7f249e7fc640 1 -- 172.21.15.96:0/236499683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24900ab550 con 0x7f24a00a4900 2024-09-13T17:45:31.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249e7fc640 1 -- 172.21.15.96:0/236499683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24900ab6b0 con 0x7f24a00a4900 2024-09-13T17:45:31.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249e7fc640 1 -- 172.21.15.96:0/236499683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24900ab970 con 0x7f24a00a4900 2024-09-13T17:45:31.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- 172.21.15.96:0/236499683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a4900 msgr2=0x7f24a00a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 --2- 172.21.15.96:0/236499683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a4900 0x7f24a00a4ce0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f2490009740 tx=0x7f249001c420 comp rx=0 tx=0).stop 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- 172.21.15.96:0/236499683 shutdown_connections 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 --2- 172.21.15.96:0/236499683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a4900 0x7f24a00a4ce0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- 172.21.15.96:0/236499683 >> 172.21.15.96:0/236499683 conn(0x7f24a006a150 msgr2=0x7f24a006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- 172.21.15.96:0/236499683 shutdown_connections 2024-09-13T17:45:31.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- 172.21.15.96:0/236499683 wait complete. 2024-09-13T17:45:31.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 Processor -- start 2024-09-13T17:45:31.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- start start 2024-09-13T17:45:31.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f24a5f13640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a006e910 con 0x7f24a00a6510 2024-09-13T17:45:31.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46030/0 (socket says 172.21.15.96:46030) 2024-09-13T17:45:31.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249f7fe640 1 -- 172.21.15.96:0/1419545920 learned_addr learned my addr 172.21.15.96:0/1419545920 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:31.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249f7fe640 1 -- 172.21.15.96:0/1419545920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490009420 con 0x7f24a00a6510 2024-09-13T17:45:31.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.842+0000 7f249f7fe640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f249001a050 tx=0x7f249001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24900bf410 con 0x7f24a00a6510 2024-09-13T17:45:31.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24900bf570 con 0x7f24a00a6510 2024-09-13T17:45:31.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a01342d0 con 0x7f24a00a6510 2024-09-13T17:45:31.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24900bf850 con 0x7f24a00a6510 2024-09-13T17:45:31.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a009ce70 con 0x7f24a00a6510 2024-09-13T17:45:31.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a009d180 con 0x7f24a00a6510 2024-09-13T17:45:31.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.846+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f24900bf9b0 con 0x7f24a00a6510 2024-09-13T17:45:31.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.850+0000 7f2487fff640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 0x7f248c037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:31.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.850+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f24900e2600 con 0x7f24a00a6510 2024-09-13T17:45:31.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.850+0000 7f249effd640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 0x7f248c037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:31.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.850+0000 7f249effd640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 0x7f248c037950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2494007fd0 tx=0x7f2494007940 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:31.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:31.854+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f24900b2b40 con 0x7f24a00a6510 2024-09-13T17:45:32.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.050+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24a013adc0 con 0x7f24a00a6510 2024-09-13T17:45:32.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f2487fff640 1 -- 172.21.15.96:0/1419545920 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f24900b2310 con 0x7f24a00a6510 2024-09-13T17:45:32.055 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:32.055 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:32.055 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:32.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 msgr2=0x7f248c037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 0x7f248c037950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2494007fd0 tx=0x7f2494007940 comp rx=0 tx=0).stop 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 msgr2=0x7f24a012e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f249001a050 tx=0x7f249001ac60 comp rx=0 tx=0).stop 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 shutdown_connections 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f248c035490 0x7f248c037950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 --2- 172.21.15.96:0/1419545920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24a00a6510 0x7f24a012e160 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 >> 172.21.15.96:0/1419545920 conn(0x7f24a006a150 msgr2=0x7f24a013b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 shutdown_connections 2024-09-13T17:45:32.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.054+0000 7f24a5f13640 1 -- 172.21.15.96:0/1419545920 wait complete. 2024-09-13T17:45:32.061 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:32.058+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:32.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-13T17:45:26.833191+0000', 'modified': '2024-09-13T17:45:29.843047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5371}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5371': {'gid': 5371, 'name': 'b', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.96:6829/1571674225', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6828', 'nonce': 1571674225}, {'type': 'v1', 'addr': '172.21.15.96:6829', 'nonce': 1571674225}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-09-13T17:45:32.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-13T17:45:32.277 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-13T17:45:32.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c6efa640 1 Processor -- start 2024-09-13T17:45:32.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c6efa640 1 -- start start 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c6efa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe4c009dc20 0x7fe4c01351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c6efa640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c006e130 con 0x7fe4c006f6d0 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c6efa640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c00a6750 con 0x7fe4c009dc20 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4bffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fe4c009dc20 0x7fe4c01351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c4c96640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fe4c006f6d0 0x7fe4c0134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44146/0 (socket says 172.21.15.96:44146) 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4c4c96640 1 -- 172.21.15.96:0/2570755943 learned_addr learned my addr 172.21.15.96:0/2570755943 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:32.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4bffff640 1 -- 172.21.15.96:0/2570755943 >> v1:172.21.15.96:6789/0 conn(0x7fe4c006f6d0 legacy=0x7fe4c0134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.414+0000 7fe4bffff640 1 -- 172.21.15.96:0/2570755943 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00712e0 con 0x7fe4c009dc20 2024-09-13T17:45:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bffff640 1 --2- 172.21.15.96:0/2570755943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c009dc20 0x7fe4c01351c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009740 tx=0x7fe4b001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01816e05aae9247 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bf7fe640 1 -- 172.21.15.96:0/2570755943 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b00ab400 con 0x7fe4c009dc20 2024-09-13T17:45:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bf7fe640 1 -- 172.21.15.96:0/2570755943 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b00a7050 con 0x7fe4c009dc20 2024-09-13T17:45:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bf7fe640 1 -- 172.21.15.96:0/2570755943 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b00ab6b0 con 0x7fe4c009dc20 2024-09-13T17:45:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/2570755943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c009dc20 msgr2=0x7fe4c01351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/2570755943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c009dc20 0x7fe4c01351c0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009740 tx=0x7fe4b001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/2570755943 shutdown_connections 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/2570755943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c009dc20 0x7fe4c01351c0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/2570755943 >> 172.21.15.96:0/2570755943 conn(0x7fe4c0098a60 msgr2=0x7fe4c009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/2570755943 shutdown_connections 2024-09-13T17:45:32.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/2570755943 wait complete. 2024-09-13T17:45:32.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 Processor -- start 2024-09-13T17:45:32.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- start start 2024-09-13T17:45:32.421 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c009a680 con 0x7fe4c006f6d0 2024-09-13T17:45:32.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51372/0 (socket says 172.21.15.96:51372) 2024-09-13T17:45:32.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c4c96640 1 -- 172.21.15.96:0/609140035 learned_addr learned my addr 172.21.15.96:0/609140035 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c4c96640 1 -- 172.21.15.96:0/609140035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b0009420 con 0x7fe4c006f6d0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c4c96640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c480 tx=0x7fe4b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bd7fa640 1 -- 172.21.15.96:0/609140035 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4005c20 con 0x7fe4c006f6d0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bd7fa640 1 -- 172.21.15.96:0/609140035 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4005d80 con 0x7fe4c006f6d0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c0098e40 con 0x7fe4c006f6d0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4bd7fa640 1 -- 172.21.15.96:0/609140035 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b40071a0 con 0x7fe4c006f6d0 2024-09-13T17:45:32.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.418+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c011d8e0 con 0x7fe4c006f6d0 2024-09-13T17:45:32.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4bd7fa640 1 -- 172.21.15.96:0/609140035 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fe4b4007300 con 0x7fe4c006f6d0 2024-09-13T17:45:32.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4bd7fa640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 0x7fe4ac0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4bd7fa640 1 -- 172.21.15.96:0/609140035 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe4b4046be0 con 0x7fe4c006f6d0 2024-09-13T17:45:32.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4bffff640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 0x7fe4ac0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4bffff640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 0x7fe4ac0379a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00ad000 tx=0x7fe4b0008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4ca043000 1 Processor -- start 2024-09-13T17:45:32.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4ca043000 1 -- start start 2024-09-13T17:45:32.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4ca043000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4ca043000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17fdabe80 con 0x55c17feea710 2024-09-13T17:45:32.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51384/0 (socket says 172.21.15.96:51384) 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4c4c96640 1 -- 172.21.15.96:0/1737457933 learned_addr learned my addr 172.21.15.96:0/1737457933 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4c4c96640 1 -- 172.21.15.96:0/1737457933 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c17fa2e890 con 0x55c17feea710 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.422+0000 7fe4c4c96640 1 --2- 172.21.15.96:0/1737457933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400cee0 tx=0x7fe4b4007670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6993cb8b25e2abb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4a67fc640 1 -- 172.21.15.96:0/1737457933 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b40365b0 con 0x55c17feea710 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4a67fc640 1 -- 172.21.15.96:0/1737457933 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b401ac40 con 0x55c17feea710 2024-09-13T17:45:32.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4a67fc640 1 -- 172.21.15.96:0/1737457933 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4011a50 con 0x55c17feea710 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/1737457933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 msgr2=0x55c17fee08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 --2- 172.21.15.96:0/1737457933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400cee0 tx=0x7fe4b4007670 comp rx=0 tx=0).stop 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/1737457933 shutdown_connections 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 --2- 172.21.15.96:0/1737457933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee08f0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/1737457933 >> 172.21.15.96:0/1737457933 conn(0x55c17fedb230 msgr2=0x55c17fee01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/1737457933 shutdown_connections 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/1737457933 wait complete. 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- start start 2024-09-13T17:45:32.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 Processor -- start 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17fd39af0 con 0x55c17feea710 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 10 client.0 _ll_register_callbacks cb 0x55c17fecfae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51390/0 (socket says 172.21.15.96:51390) 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4c4c96640 1 -- 172.21.15.96:0/3050353679 learned_addr learned my addr 172.21.15.96:0/3050353679 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4c4c96640 1 -- 172.21.15.96:0/3050353679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c011d8e0 con 0x55c17feea710 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4c4c96640 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4039040 tx=0x7fe4b4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3c817409bc793b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4045c40 con 0x55c17feea710 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe4b4045da0 con 0x55c17feea710 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c17fcef5f0 con 0x55c17feea710 2024-09-13T17:45:32.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4b4010ae0 con 0x55c17feea710 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 10 client.5430 fetch_fsmap learned FSMap version 43 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55c17fcf7ab0 con 0x55c17feea710 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c17fcef5f0 con 0x55c17feea710 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) v1 ==== 2024+0+0 (secure 0 0 0) 0x7fe4b4031070 con 0x55c17feea710 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 10 client.5430 fetch_fsmap finished waiting for FSMap version 43 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 10 client.5430 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 20 client.5430 populate_metadata read hostname 'smithi096' 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17fef0130 0x55c17ff16b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 4 client.5430 mds_command: new command op to 5371 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 --> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c17fd2f790 con 0x55c17fef0130 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4bffff640 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17fef0130 0x55c17ff16b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe49f7fe640 10 client.5430 ms_handle_connect on v2:172.21.15.96:6828/1571674225 2024-09-13T17:45:32.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.426+0000 7fe4bffff640 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17fef0130 0x55c17ff16b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a5fb1dd37c4c0c server_cookie=d1a1b8385a07915a in_seq=0 out_seq=0 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3050353679 <== mds.0 v2:172.21.15.96:6828/1571674225 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19071 (crc 0 0 0) 0x55c17fd2f790 con 0x55c17fef0130 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe49f7fe640 10 client.5430 handle_command_reply: tid=0 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 client.5430 shutdown 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 msgr2=0x55c17ff10760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4039040 tx=0x7fe4b4005ef0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 shutdown_connections 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17fef0130 0x55c17ff16b20 crc :-1 s=READY pgs=6 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3050353679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff10760 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe49f7fe640 0 client.5430 ms_handle_reset on v2:172.21.15.96:6828/1571674225 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 >> 172.21.15.96:0/3050353679 conn(0x55c17fedb230 msgr2=0x55c17fee3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 shutdown_connections 2024-09-13T17:45:32.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 1 -- 172.21.15.96:0/3050353679 wait complete. 2024-09-13T17:45:32.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.430+0000 7fe4ca043000 20 client.5430 trim_cache size 0 max 16384 2024-09-13T17:45:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 Processor -- start 2024-09-13T17:45:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- start start 2024-09-13T17:45:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17fd370c0 con 0x55c17feea710 2024-09-13T17:45:32.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51392/0 (socket says 172.21.15.96:51392) 2024-09-13T17:45:32.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4c4c96640 1 -- 172.21.15.96:0/2925594489 learned_addr learned my addr 172.21.15.96:0/2925594489 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4c4c96640 1 -- 172.21.15.96:0/2925594489 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c17fcddc00 con 0x55c17feea710 2024-09-13T17:45:32.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4c4c96640 1 --2- 172.21.15.96:0/2925594489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40109e0 tx=0x7fe4b4024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc286dd993ea23ce server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4a6ffd640 1 -- 172.21.15.96:0/2925594489 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4011a50 con 0x55c17feea710 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4a6ffd640 1 -- 172.21.15.96:0/2925594489 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b4024070 con 0x55c17feea710 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4a6ffd640 1 -- 172.21.15.96:0/2925594489 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4011ce0 con 0x55c17feea710 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- 172.21.15.96:0/2925594489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 msgr2=0x55c17fee9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 --2- 172.21.15.96:0/2925594489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40109e0 tx=0x7fe4b4024000 comp rx=0 tx=0).stop 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- 172.21.15.96:0/2925594489 shutdown_connections 2024-09-13T17:45:32.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 --2- 172.21.15.96:0/2925594489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17fee9600 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- 172.21.15.96:0/2925594489 >> 172.21.15.96:0/2925594489 conn(0x55c17fedb230 msgr2=0x55c17fee8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- 172.21.15.96:0/2925594489 shutdown_connections 2024-09-13T17:45:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- 172.21.15.96:0/2925594489 wait complete. 2024-09-13T17:45:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 -- start start 2024-09-13T17:45:32.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.446+0000 7fe4ca043000 1 Processor -- start 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17fd329b0 con 0x55c17feea710 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 10 client.0 _ll_register_callbacks cb 0x55c17fce1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c4c96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51396/0 (socket says 172.21.15.96:51396) 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c4c96640 1 -- 172.21.15.96:0/3547541959 learned_addr learned my addr 172.21.15.96:0/3547541959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c4c96640 1 -- 172.21.15.96:0/3547541959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c17fcf7ab0 con 0x55c17feea710 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c4c96640 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40076b0 tx=0x7fe4b4010380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a515bd13580c1704 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-13T17:45:32.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe4b4045e50 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fe4b4010ae0 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c17fd02460 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4b4010ce0 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 10 client.5432 fetch_fsmap learned FSMap version 43 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x55c17fa2ecc0 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c17fd02460 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) v1 ==== 2024+0+0 (secure 0 0 0) 0x7fe4b4031040 con 0x55c17feea710 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 10 client.5432 fetch_fsmap finished waiting for FSMap version 43 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 10 client.5432 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 20 client.5432 populate_metadata read hostname 'smithi096' 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17ff485d0 0x55c17ff489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 4 client.5432 mds_command: new command op to 5371 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 --> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c17fcfb9c0 con 0x55c17ff485d0 2024-09-13T17:45:32.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4bffff640 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17ff485d0 0x55c17ff489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4bffff640 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17ff485d0 0x55c17ff489b0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95be735bbe17d1bd server_cookie=18485c0e8c066240 in_seq=0 out_seq=0 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 10 client.5432 ms_handle_connect on v2:172.21.15.96:6828/1571674225 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 1 -- 172.21.15.96:0/3547541959 <== mds.0 v2:172.21.15.96:6828/1571674225 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c17fcfb9c0 con 0x55c17ff485d0 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 10 client.5432 handle_command_reply: tid=0 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 client.5432 shutdown 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 msgr2=0x55c17ff48090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40076b0 tx=0x7fe4b4010380 comp rx=0 tx=0).stop 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 shutdown_connections 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:6828/1571674225,v1:172.21.15.96:6829/1571674225] conn(0x55c17ff485d0 0x55c17ff489b0 crc :-1 s=READY pgs=7 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 --2- 172.21.15.96:0/3547541959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c17feea710 0x55c17ff48090 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe49f7fe640 0 client.5432 ms_handle_reset on v2:172.21.15.96:6828/1571674225 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 >> 172.21.15.96:0/3547541959 conn(0x55c17fedb230 msgr2=0x55c17fee9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 shutdown_connections 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 1 -- 172.21.15.96:0/3547541959 wait complete. 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4ca043000 20 client.5432 trim_cache size 0 max 16384 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "a54aa7b1-da4c-43a3-b94f-1607e77cbcc4", 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5371, 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 5.6177494960000001, 2024-09-13T17:45:32.457 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 43, 2024-09-13T17:45:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 68, 2024-09-13T17:45:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 68, 2024-09-13T17:45:32.458 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 10.070870897000001 2024-09-13T17:45:32.458 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 msgr2=0x7fe4ac0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 0x7fe4ac0379a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4b00ad000 tx=0x7fe4b0008040 comp rx=0 tx=0).stop 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 msgr2=0x7fe4c01cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400c480 tx=0x7fe4b400c940 comp rx=0 tx=0).stop 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 shutdown_connections 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fe4ac0354e0 0x7fe4ac0379a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 --2- 172.21.15.96:0/609140035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c006f6d0 0x7fe4c01cfa10 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 >> 172.21.15.96:0/609140035 conn(0x7fe4c0098a60 msgr2=0x7fe4c006fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 shutdown_connections 2024-09-13T17:45:32.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.450+0000 7fe4c6efa640 1 -- 172.21.15.96:0/609140035 wait complete. 2024-09-13T17:45:32.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:45:32.585 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a1115a640 1 Processor -- start 2024-09-13T17:45:32.586 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a1115a640 1 -- start start 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a1115a640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a0c06ac90 0x7f0a0c09e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a1115a640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0c09f0c0 con 0x7f0a0c09ac70 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a1115a640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0c09cde0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a0a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a0c06ac90 0x7f0a0c09e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a0a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0a0c06ac90 0x7f0a0c09e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51398/0 (socket says 172.21.15.96:51398) 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a0a59c640 1 -- 172.21.15.96:0/38306946 learned_addr learned my addr 172.21.15.96:0/38306946 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.582+0000 7f0a0ad9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f0a0c09ac70 0x7f0a0c09b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44158/0 (socket says 172.21.15.96:44158) 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a0a59c640 1 -- 172.21.15.96:0/38306946 >> v1:172.21.15.96:6789/0 conn(0x7f0a0c09ac70 legacy=0x7f0a0c09b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a09d9b640 1 -- 172.21.15.96:0/38306946 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021101430 0 0) 0x7f0a0c09f0c0 con 0x7f0a0c09ac70 2024-09-13T17:45:32.587 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a0a59c640 1 -- 172.21.15.96:0/38306946 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c09be10 con 0x7f0a0c06ac90 2024-09-13T17:45:32.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a0a59c640 1 --2- 172.21.15.96:0/38306946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c09e970 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097f0 tx=0x7f0a0001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b992a7d13047829a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a09d9b640 1 -- 172.21.15.96:0/38306946 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a000ab530 con 0x7f0a0c06ac90 2024-09-13T17:45:32.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a09d9b640 1 -- 172.21.15.96:0/38306946 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000a7050 con 0x7f0a0c06ac90 2024-09-13T17:45:32.589 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a09d9b640 1 -- 172.21.15.96:0/38306946 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a000ab7e0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 -- 172.21.15.96:0/38306946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 msgr2=0x7f0a0c09e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 --2- 172.21.15.96:0/38306946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c09e970 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097f0 tx=0x7f0a0001c450 comp rx=0 tx=0).stop 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 -- 172.21.15.96:0/38306946 shutdown_connections 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 --2- 172.21.15.96:0/38306946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c09e970 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 -- 172.21.15.96:0/38306946 >> 172.21.15.96:0/38306946 conn(0x7f0a0c06e350 msgr2=0x7f0a0c070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 -- 172.21.15.96:0/38306946 shutdown_connections 2024-09-13T17:45:32.590 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.586+0000 7f0a1115a640 1 -- 172.21.15.96:0/38306946 wait complete. 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 Processor -- start 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 -- start start 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c0a3d80 con 0x7f0a0c06ac90 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51408/0 (socket says 172.21.15.96:51408) 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0ad9d640 1 -- 172.21.15.96:0/1114613631 learned_addr learned my addr 172.21.15.96:0/1114613631 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:32.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0ad9d640 1 -- 172.21.15.96:0/1114613631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a000094d0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0ad9d640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c420 tx=0x7f09fc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09fc0074d0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc007630 con 0x7f0a0c06ac90 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0c0a2910 con 0x7f0a0c06ac90 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09fc004960 con 0x7f0a0c06ac90 2024-09-13T17:45:32.593 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c0a47a0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0c0a25c0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f09fc004ac0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 0x7f09f80379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f09fc046ce0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0a59c640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 0x7f09f80379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:32.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.590+0000 7f0a0a59c640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 0x7f09f80379a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097f0 tx=0x7f0a00008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:32.598 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.594+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f09fc01acd0 con 0x7f0a0c06ac90 2024-09-13T17:45:32.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a0c0a8200 con 0x7f0a0c06ac90 2024-09-13T17:45:32.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f09f37fe640 1 -- 172.21.15.96:0/1114613631 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f09fc018020 con 0x7f0a0c06ac90 2024-09-13T17:45:32.804 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 msgr2=0x7f09f80379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 0x7f09f80379a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0a000097f0 tx=0x7f0a00008040 comp rx=0 tx=0).stop 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 msgr2=0x7f0a0c0a4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c420 tx=0x7f09fc00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 shutdown_connections 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f09f80354e0 0x7f09f80379a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 --2- 172.21.15.96:0/1114613631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0a0c06ac90 0x7f0a0c0a4f20 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 >> 172.21.15.96:0/1114613631 conn(0x7f0a0c06e350 msgr2=0x7f0a0c06fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 shutdown_connections 2024-09-13T17:45:32.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:32.802+0000 7f0a1115a640 1 -- 172.21.15.96:0/1114613631 wait complete. 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:45:32.823 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:45:32.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:32.824 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:45:32.829 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:45:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:45:32.829 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:45:32.830 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42349sec preferred_lft 42349sec 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:45:32.831 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:32.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:32.833 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:45:32.881 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:45:32.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:32.883 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:45:32.922 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:32.918+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:32.930 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:45:32.930 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:45:32.930 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:45:32.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:32.931 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:45:32.931 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:45:32.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:45:32.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:45:32.932 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:45:32.932 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:45:33.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:33.013 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:45:33.061 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:45:33.061 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:45:33.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:45:33.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:33.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-09-13T17:45:33.115 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-13T17:45:33.115 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:45:33.116 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:33.165 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-13T17:45:33.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:45:33.165 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:33.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:33.269 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:33.269 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:33.323 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:33.324 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:33.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:33.325 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:33.375 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-13T17:45:33.376 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-13T17:45:33.420 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:33.434 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:33.434 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:33.489 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:33.490 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:33.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:33.491 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:33.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:2024-09-13T17:45:33.498+0000 7f8a17b78580 -1 init, newargv = 0x558da8b9b7d0 newargc=15 2024-09-13T17:45:33.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[44241]: starting ceph client 2024-09-13T17:45:33.507 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[44241]: starting fuse 2024-09-13T17:45:33.801 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:33.798+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:34.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:34.624 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:34.625 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:34.683 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:34.684 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:34.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:34.685 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:34.732 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:45:34.732 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:34.733 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.0")) 2024-09-13T17:45:34.734 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:45:34.801 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:34.798+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:34.857 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.0.44241.asok 2024-09-13T17:45:35.037 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44241.asok 2024-09-13T17:45:35.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.038 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44241.asok status 2024-09-13T17:45:35.205 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:35.205 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "0", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "44241", 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5438, 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:45:35.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 5438 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:45:35.207 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 245796833 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.1:0", 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 245796833 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.5438 192.168.0.1:0/245796833", 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.1:0/245796833", 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 43, 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 68, 2024-09-13T17:45:35.208 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:45:35.209 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:45:35.209 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:45:35.209 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:35.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.220 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:45:35.220 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:35.225 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:45:35.226 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:35.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.226 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:35.281 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-13T17:45:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.450+0000 7fba1e583640 1 Processor -- start 2024-09-13T17:45:35.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.450+0000 7fba1e583640 1 -- start start 2024-09-13T17:45:35.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba1e583640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fba1806faa0 0x7fba180a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba1e583640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1806eee0 con 0x7fba1806f6c0 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba1e583640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba18070090 con 0x7fba1806faa0 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba177fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fba1806faa0 0x7fba180a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba17fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fba1806f6c0 0x7fba180a1470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44170/0 (socket says 172.21.15.96:44170) 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba17fff640 1 -- 172.21.15.96:0/2699442222 learned_addr learned my addr 172.21.15.96:0/2699442222 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba16ffd640 1 -- 172.21.15.96:0/2699442222 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409246150 0 0) 0x7fba1806eee0 con 0x7fba1806f6c0 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba16ffd640 1 -- 172.21.15.96:0/2699442222 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba00003450 con 0x7fba1806f6c0 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba177fe640 1 -- 172.21.15.96:0/2699442222 >> v1:172.21.15.96:6789/0 conn(0x7fba1806f6c0 legacy=0x7fba180a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:35.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba177fe640 1 -- 172.21.15.96:0/2699442222 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180a3040 con 0x7fba1806faa0 2024-09-13T17:45:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba177fe640 1 --2- 172.21.15.96:0/2699442222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806faa0 0x7fba180a1b80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009770 tx=0x7fba0c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac65abffdca4ede server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba16ffd640 1 -- 172.21.15.96:0/2699442222 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba0c0ab530 con 0x7fba1806faa0 2024-09-13T17:45:35.459 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba16ffd640 1 -- 172.21.15.96:0/2699442222 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c0a7050 con 0x7fba1806faa0 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.454+0000 7fba16ffd640 1 -- 172.21.15.96:0/2699442222 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba0c0ab7e0 con 0x7fba1806faa0 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/2699442222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806faa0 msgr2=0x7fba180a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 --2- 172.21.15.96:0/2699442222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806faa0 0x7fba180a1b80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009770 tx=0x7fba0c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/2699442222 shutdown_connections 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 --2- 172.21.15.96:0/2699442222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806faa0 0x7fba180a1b80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/2699442222 >> 172.21.15.96:0/2699442222 conn(0x7fba1806a150 msgr2=0x7fba1806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:35.460 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/2699442222 shutdown_connections 2024-09-13T17:45:35.461 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/2699442222 wait complete. 2024-09-13T17:45:35.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 Processor -- start 2024-09-13T17:45:35.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- start start 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1811e4f0 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51412/0 (socket says 172.21.15.96:51412) 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba17fff640 1 -- 172.21.15.96:0/1838984383 learned_addr learned my addr 172.21.15.96:0/1838984383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba17fff640 1 -- 172.21.15.96:0/1838984383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c009420 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba17fff640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c3a0 tx=0x7fba0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08005c20 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba08005d80 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1809ffe0 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba08007220 con 0x7fba1806f6c0 2024-09-13T17:45:35.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1811e0a0 con 0x7fba1806f6c0 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.458+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1811e2c0 con 0x7fba1806f6c0 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.462+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fba08007380 con 0x7fba1806f6c0 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.462+0000 7fba14ff9640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 0x7fba040379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.462+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7fba08046d00 con 0x7fba1806f6c0 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.462+0000 7fba177fe640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 0x7fba040379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:35.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.462+0000 7fba177fe640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 0x7fba040379a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0ad000 tx=0x7fba0c008040 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:35.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.466+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fba08019b40 con 0x7fba1806f6c0 2024-09-13T17:45:35.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.674+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fba181388e0 con 0x7fba1806f6c0 2024-09-13T17:45:35.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.674+0000 7fba14ff9640 1 -- 172.21.15.96:0/1838984383 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7fba08017020 con 0x7fba1806f6c0 2024-09-13T17:45:35.677 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 msgr2=0x7fba040379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 0x7fba040379a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0ad000 tx=0x7fba0c008040 comp rx=0 tx=0).stop 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 msgr2=0x7fba1809e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c3a0 tx=0x7fba0800c860 comp rx=0 tx=0).stop 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 shutdown_connections 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fba040354e0 0x7fba040379a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 --2- 172.21.15.96:0/1838984383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba1806f6c0 0x7fba1809e090 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 >> 172.21.15.96:0/1838984383 conn(0x7fba1806a150 msgr2=0x7fba18134a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 shutdown_connections 2024-09-13T17:45:35.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:35.678+0000 7fba1e583640 1 -- 172.21.15.96:0/1838984383 wait complete. 2024-09-13T17:45:35.697 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-13T17:45:35.697 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-13T17:45:35.697 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-13T17:45:35.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:45:35.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.698 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-09-13T17:45:35.704 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-13T17:45:35.704 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-13T17:45:35.704 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-09-13T17:45:35.704 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.704 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-13T17:45:35.705 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42347sec preferred_lft 42347sec 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::ec4:7aff:fe8f:cc0d/64 scope link 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 02:42:bd:9d:cf:1e brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:45:35.706 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-09-13T17:45:35.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.707 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-13T17:45:35.728 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:35.726+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:35.740 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-09-13T17:45:35.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.741 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-09-13T17:45:35.790 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-09-13T17:45:35.790 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-13T17:45:35.790 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-13T17:45:35.790 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-13T17:45:35.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.791 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec bash -c ' 2024-09-13T17:45:35.791 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-09-13T17:45:35.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-13T17:45:35.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-13T17:45:35.792 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-13T17:45:35.792 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-09-13T17:45:35.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.869 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:45:35.915 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:45:35.915 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:45:35.915 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:45:35.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:35.916 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.1 up 2024-09-13T17:45:35.971 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-13T17:45:35.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:45:35.972 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:36.020 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-13T17:45:36.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-13T17:45:36.020 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:36.068 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:36.123 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:36.124 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:36.179 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:36.180 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:36.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:36.181 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:36.227 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:45:36.228 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-13T17:45:36.228 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-13T17:45:36.276 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:36.291 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:36.291 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:36.344 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:36.345 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:36.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:36.346 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:36.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:2024-09-13T17:45:36.354+0000 7f9b22469580 -1 init, newargv = 0x55611dd07390 newargc=15 2024-09-13T17:45:36.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[44348]: starting ceph client 2024-09-13T17:45:36.360 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:45:36.361 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[44348]: starting fuse 2024-09-13T17:45:36.614 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:36.610+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:37.462 DEBUG:teuthology.orchestra.run.smithi096:> sudo modprobe fuse 2024-09-13T17:45:37.477 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-13T17:45:37.477 DEBUG:teuthology.orchestra.run.smithi096:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-09-13T17:45:37.531 INFO:teuthology.orchestra.run.smithi096.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-09-13T17:45:37.532 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-09-13T17:45:37.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:37.533 DEBUG:teuthology.orchestra.run.smithi096:> ls /sys/fs/fuse/connections 2024-09-13T17:45:37.579 INFO:teuthology.orchestra.run.smithi096.stdout:45 2024-09-13T17:45:37.579 INFO:teuthology.orchestra.run.smithi096.stdout:46 2024-09-13T17:45:37.580 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> import glob 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> import re 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> import os 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> import subprocess 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> def _find_admin_socket(client_name): 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> files = glob.glob(asok_path) 2024-09-13T17:45:37.580 DEBUG:teuthology.orchestra.run.smithi096:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> # Given a non-glob path, it better be there 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> if "*" not in asok_path: 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> assert(len(files) == 1) 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> return files[0] 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> for f in files: 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> contents = proc_f.read() 2024-09-13T17:45:37.581 DEBUG:teuthology.orchestra.run.smithi096:> if mountpoint in contents: 2024-09-13T17:45:37.582 DEBUG:teuthology.orchestra.run.smithi096:> return f 2024-09-13T17:45:37.582 DEBUG:teuthology.orchestra.run.smithi096:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-13T17:45:37.582 DEBUG:teuthology.orchestra.run.smithi096:> 2024-09-13T17:45:37.582 DEBUG:teuthology.orchestra.run.smithi096:> print(_find_admin_socket("client.1")) 2024-09-13T17:45:37.582 DEBUG:teuthology.orchestra.run.smithi096:> ' 2024-09-13T17:45:37.636 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:37.634+0000 7f304da43640 -1 client.5292: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5247:cc0ce67a-5ee7-4094-951e-63820f385330:v1:172.21.15.96:0/1206016791] 2024-09-13T17:45:37.707 INFO:teuthology.orchestra.run.smithi096.stdout:/var/run/ceph/ceph-client.1.44348.asok 2024-09-13T17:45:37.888 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44348.asok 2024-09-13T17:45:37.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:37.889 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44348.asok status 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "metadata": { 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_sha1": "6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3", 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 18.2.4-860-g6c6b083f (6c6b083f148f500c4f56d402b9e4ffcab6f1e2d3) reef (stable)", 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "entity_id": "1", 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": "smithi096", 2024-09-13T17:45:38.054 INFO:teuthology.orchestra.run.smithi096.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "pid": "44348", 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "root": "/" 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_count": 0, 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "dentry_pinned_count": 0, 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 5444, 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "inst": { 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": { 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "client", 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "num": 5444 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:38.055 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3896162559 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": { 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "192.168.0.2:0", 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3896162559 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "inst_str": "client.5444 192.168.0.2:0/3896162559", 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "addr_str": "192.168.0.2:0/3896162559", 2024-09-13T17:45:38.056 INFO:teuthology.orchestra.run.smithi096.stdout: "inode_count": 1, 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_epoch": 43, 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch": 68, 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_epoch_barrier": 0, 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "blocklisted": false, 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs" 2024-09-13T17:45:38.057 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:38.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:38.069 DEBUG:teuthology.orchestra.run.smithi096:> stat --file-system '--printf=%T 2024-09-13T17:45:38.070 DEBUG:teuthology.orchestra.run.smithi096:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:38.076 INFO:teuthology.orchestra.run.smithi096.stdout:fuseblk 2024-09-13T17:45:38.076 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:38.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:38.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:38.131 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable stats 2024-09-13T17:45:38.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e9e9a4640 1 Processor -- start 2024-09-13T17:45:38.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e9e9a4640 1 -- start start 2024-09-13T17:45:38.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e9e9a4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e98134e70 0x7f5e9809e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:38.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e9e9a4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9809ed60 con 0x7f5e98135250 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e9e9a4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9809ddc0 con 0x7f5e98134e70 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e97fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e98134e70 0x7f5e9809e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e97fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e98134e70 0x7f5e9809e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51420/0 (socket says 172.21.15.96:51420) 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e977fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5e98135250 0x7f5e98135660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44176/0 (socket says 172.21.15.96:44176) 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e97fff640 1 -- 172.21.15.96:0/1103155684 learned_addr learned my addr 172.21.15.96:0/1103155684 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e97fff640 1 -- 172.21.15.96:0/1103155684 >> v1:172.21.15.96:6789/0 conn(0x7f5e98135250 legacy=0x7f5e98135660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e97fff640 1 -- 172.21.15.96:0/1103155684 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980a68b0 con 0x7f5e98134e70 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.298+0000 7f5e96ffd640 1 -- 172.21.15.96:0/1103155684 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388612573 0 0) 0x7f5e8c001120 con 0x7f5e98135250 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e97fff640 1 --2- 172.21.15.96:0/1103155684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e9809e820 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5e880097f0 tx=0x7f5e8801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10360375536272bf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e96ffd640 1 -- 172.21.15.96:0/1103155684 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e880ac540 con 0x7f5e98134e70 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e96ffd640 1 -- 172.21.15.96:0/1103155684 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e880ac6a0 con 0x7f5e98134e70 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e96ffd640 1 -- 172.21.15.96:0/1103155684 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e880ac960 con 0x7f5e98134e70 2024-09-13T17:45:38.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/1103155684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 msgr2=0x7f5e9809e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/1103155684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e9809e820 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f5e880097f0 tx=0x7f5e8801c4e0 comp rx=0 tx=0).stop 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/1103155684 shutdown_connections 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/1103155684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e9809e820 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/1103155684 >> 172.21.15.96:0/1103155684 conn(0x7f5e98098a60 msgr2=0x7f5e9809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/1103155684 shutdown_connections 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/1103155684 wait complete. 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 Processor -- start 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- start start 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e9e9a4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980a2fe0 con 0x7f5e98134e70 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:38.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e97fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51432/0 (socket says 172.21.15.96:51432) 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.302+0000 7f5e97fff640 1 -- 172.21.15.96:0/3235120406 learned_addr learned my addr 172.21.15.96:0/3235120406 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e97fff640 1 -- 172.21.15.96:0/3235120406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e880094d0 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e97fff640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801a110 tx=0x7f5e8801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88004160 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e880042c0 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e980a0f20 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e88003620 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e980a1d50 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e981368b0 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 32) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5e880b3040 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 0x7f5e840379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 3439+0+0 (secure 0 0 0) 0x7f5e880e1d10 con 0x7f5e98134e70 2024-09-13T17:45:38.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e977fe640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 0x7f5e840379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:38.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.306+0000 7f5e977fe640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 0x7f5e840379a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c10 tx=0x7f5e8c001320 comp rx=0 tx=0).ready entity=mgr.5260 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:38.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.310+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e880c7530 con 0x7f5e98134e70 2024-09-13T17:45:38.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.510+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "stats"} v 0) v1 -- 0x7f5e980a0440 con 0x7f5e98134e70 2024-09-13T17:45:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.878+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 33) v1 ==== 43362+0+0 (secure 0 0 0) 0x7f5e88003ca0 con 0x7f5e98134e70 2024-09-13T17:45:38.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.878+0000 7f5e94ff9640 1 -- 172.21.15.96:0/3235120406 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "stats"}]=0 v33) v1 ==== 84+0+0 (secure 0 0 0) 0x7f5e880c8ae0 con 0x7f5e98134e70 2024-09-13T17:45:38.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 msgr2=0x7f5e840379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:38.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 0x7f5e840379a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e8c000c10 tx=0x7f5e8c001320 comp rx=0 tx=0).stop 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 msgr2=0x7f5e980a27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f5e8801a110 tx=0x7f5e8801ad20 comp rx=0 tx=0).stop 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 shutdown_connections 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f5e840354e0 0x7f5e840379a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 --2- 172.21.15.96:0/3235120406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e98134e70 0x7f5e980a27b0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 >> 172.21.15.96:0/3235120406 conn(0x7f5e98098a60 msgr2=0x7f5e98099380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 shutdown_connections 2024-09-13T17:45:38.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:38.882+0000 7f5e9e9a4640 1 -- 172.21.15.96:0/3235120406 wait complete. 2024-09-13T17:45:38.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:38.909 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec cephfs-top --cluster=hpec --id=admin --selftest) 2024-09-13T17:45:38.972 INFO:teuthology.orchestra.run.smithi096.stderr:cluster hpec does not exist 2024-09-13T17:45:38.979 DEBUG:teuthology.orchestra.run:got remote process result: 255 2024-09-13T17:45:38.980 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable stats 2024-09-13T17:45:39.017 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:39.014+0000 7fbfc0896180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:45:39.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c424e640 1 Processor -- start 2024-09-13T17:45:39.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c424e640 1 -- start start 2024-09-13T17:45:39.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c424e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc5bc098a60 0x7fc5bc098e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c424e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5bc09c020 con 0x7fc5bc09a810 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c424e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5bc09cc90 con 0x7fc5bc098a60 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c1fea640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc5bc098a60 0x7fc5bc098e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c17e9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc5bc09a810 0x7fc5bc134cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44190/0 (socket says 172.21.15.96:44190) 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c17e9640 1 -- 172.21.15.96:0/4163930777 learned_addr learned my addr 172.21.15.96:0/4163930777 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c1fea640 1 -- 172.21.15.96:0/4163930777 >> v1:172.21.15.96:6789/0 conn(0x7fc5bc09a810 legacy=0x7fc5bc134cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c1fea640 1 -- 172.21.15.96:0/4163930777 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5bc09c2e0 con 0x7fc5bc098a60 2024-09-13T17:45:39.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.094+0000 7fc5c0fe8640 1 -- 172.21.15.96:0/4163930777 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570227028 0 0) 0x7fc5bc09c020 con 0x7fc5bc09a810 2024-09-13T17:45:39.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c1fea640 1 --2- 172.21.15.96:0/4163930777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc098e40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971270f39b23a764 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:39.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c0fe8640 1 -- 172.21.15.96:0/4163930777 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b40a5650 con 0x7fc5bc098a60 2024-09-13T17:45:39.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c0fe8640 1 -- 172.21.15.96:0/4163930777 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b40a57b0 con 0x7fc5bc098a60 2024-09-13T17:45:39.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c0fe8640 1 -- 172.21.15.96:0/4163930777 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b40a5a70 con 0x7fc5bc098a60 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- 172.21.15.96:0/4163930777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 msgr2=0x7fc5bc098e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 --2- 172.21.15.96:0/4163930777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc098e40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4009d90 tx=0x7fc5b401c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- 172.21.15.96:0/4163930777 shutdown_connections 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 --2- 172.21.15.96:0/4163930777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc098e40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- 172.21.15.96:0/4163930777 >> 172.21.15.96:0/4163930777 conn(0x7fc5bc06e350 msgr2=0x7fc5bc070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- 172.21.15.96:0/4163930777 shutdown_connections 2024-09-13T17:45:39.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- 172.21.15.96:0/4163930777 wait complete. 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 Processor -- start 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- start start 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c424e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5bc11f9e0 con 0x7fc5bc098a60 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c1fea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:39.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c1fea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51488/0 (socket says 172.21.15.96:51488) 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.098+0000 7fc5c1fea640 1 -- 172.21.15.96:0/2236049116 learned_addr learned my addr 172.21.15.96:0/2236049116 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c1fea640 1 -- 172.21.15.96:0/2236049116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b4009a70 con 0x7fc5bc098a60 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c1fea640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40194f0 tx=0x7fc5b4019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b40ad650 con 0x7fc5bc098a60 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b40ad7b0 con 0x7fc5bc098a60 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5bc120a30 con 0x7fc5bc098a60 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc5b40ada50 con 0x7fc5bc098a60 2024-09-13T17:45:39.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5bc120c60 con 0x7fc5bc098a60 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5bc11f100 con 0x7fc5bc098a60 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 33) v1 ==== 43362+0+0 (secure 0 0 0) 0x7fc5b40b4050 con 0x7fc5bc098a60 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fc5b40e26c0 con 0x7fc5bc098a60 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c17e9640 1 -- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 msgr2=0x7fc5a00379f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:39.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.102+0000 7fc5c17e9640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:39.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.106+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc5b40c19a0 con 0x7fc5bc098a60 2024-09-13T17:45:39.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.294+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "stats"} v 0) v1 -- 0x7fc5bc098e40 con 0x7fc5bc098a60 2024-09-13T17:45:39.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.306+0000 7fc5c17e9640 1 -- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 msgr2=0x7fc5a00379f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:39.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.306+0000 7fc5c17e9640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:45:39.397 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:39.394+0000 7fbfc0896180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:45:39.525 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:39.522+0000 7fbfc0896180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:45:39.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.710+0000 7fc5c17e9640 1 -- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 msgr2=0x7fc5a00379f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:39.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.710+0000 7fc5c17e9640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-13T17:45:39.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.882+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fc5b40b7410 con 0x7fc5bc098a60 2024-09-13T17:45:39.885 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.882+0000 7fc5ae7fc640 1 -- 172.21.15.96:0/2236049116 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "stats"}]=0 v34) v1 ==== 85+0+0 (secure 0 0 0) 0x7fc5b40c0760 con 0x7fc5bc098a60 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 msgr2=0x7fc5a00379f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 msgr2=0x7fc5bc11db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40194f0 tx=0x7fc5b4019680 comp rx=0 tx=0).stop 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 shutdown_connections 2024-09-13T17:45:39.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc5a0035530 0x7fc5a00379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 --2- 172.21.15.96:0/2236049116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5bc098a60 0x7fc5bc11db50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 >> 172.21.15.96:0/2236049116 conn(0x7fc5bc06e350 msgr2=0x7fc5bc13a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 shutdown_connections 2024-09-13T17:45:39.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:39.886+0000 7fc5c424e640 1 -- 172.21.15.96:0/2236049116 wait complete. 2024-09-13T17:45:39.907 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-13T17:45:39.907 DEBUG:teuthology.orchestra.run.smithi096:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-13T17:45:39.944 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:39.942+0000 7fbfc0896180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:39.995 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:39.994+0000 7fbfc0896180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:45:40.004 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-13T17:45:40.004 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:40.004 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:40.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:40.045 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.042+0000 7fbfc0896180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:45:40.121 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.118+0000 7fbfc0896180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:45:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.150+0000 7f94473c1640 1 Processor -- start 2024-09-13T17:45:40.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.150+0000 7f94473c1640 1 -- start start 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f944013af50 0x7f944013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440138050 con 0x7f944009d690 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944009c410 con 0x7f944013af50 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944515d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f944009d690 0x7f9440138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44204/0 (socket says 172.21.15.96:44204) 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944515d640 1 -- 172.21.15.96:0/1510374319 learned_addr learned my addr 172.21.15.96:0/1510374319 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944495c640 1 --2- 172.21.15.96:0/1510374319 >> v2:172.21.15.96:3300/0 conn(0x7f944013af50 0x7f944013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944495c640 1 -- 172.21.15.96:0/1510374319 >> v1:172.21.15.96:6789/0 conn(0x7f944009d690 legacy=0x7f9440138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944495c640 1 -- 172.21.15.96:0/1510374319 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944009cba0 con 0x7f944013af50 2024-09-13T17:45:40.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f9437fff640 1 -- 172.21.15.96:0/1510374319 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659846336 0 0) 0x7f943c001120 con 0x7f944009d690 2024-09-13T17:45:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f944495c640 1 --2- 172.21.15.96:0/1510374319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944013af50 0x7f944013d340 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9430009740 tx=0x7f943001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d3b39be61e3840 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f9437fff640 1 -- 172.21.15.96:0/1510374319 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94300a8840 con 0x7f944013af50 2024-09-13T17:45:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f9437fff640 1 -- 172.21.15.96:0/1510374319 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94300a89a0 con 0x7f944013af50 2024-09-13T17:45:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f9437fff640 1 -- 172.21.15.96:0/1510374319 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f94300a8c60 con 0x7f944013af50 2024-09-13T17:45:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- 172.21.15.96:0/1510374319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944013af50 msgr2=0x7f944013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 --2- 172.21.15.96:0/1510374319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944013af50 0x7f944013d340 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9430009740 tx=0x7f943001ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:40.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- 172.21.15.96:0/1510374319 shutdown_connections 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 --2- 172.21.15.96:0/1510374319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944013af50 0x7f944013d340 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- 172.21.15.96:0/1510374319 >> 172.21.15.96:0/1510374319 conn(0x7f944006bc60 msgr2=0x7f944006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- 172.21.15.96:0/1510374319 shutdown_connections 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.154+0000 7f94473c1640 1 -- 172.21.15.96:0/1510374319 wait complete. 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 Processor -- start 2024-09-13T17:45:40.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 -- start start 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400a6280 con 0x7f944009d690 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944515d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944515d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51510/0 (socket says 172.21.15.96:51510) 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944515d640 1 -- 172.21.15.96:0/3026807263 learned_addr learned my addr 172.21.15.96:0/3026807263 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:40.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944515d640 1 -- 172.21.15.96:0/3026807263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9430009420 con 0x7f944009d690 2024-09-13T17:45:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944515d640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c390 tx=0x7f943c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f943c007410 con 0x7f944009d690 2024-09-13T17:45:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c007590 con 0x7f944009d690 2024-09-13T17:45:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94400a51d0 con 0x7f944009d690 2024-09-13T17:45:40.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f943c0047f0 con 0x7f944009d690 2024-09-13T17:45:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94400a7170 con 0x7f944009d690 2024-09-13T17:45:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94400a5ee0 con 0x7f944009d690 2024-09-13T17:45:40.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f943c004950 con 0x7f944009d690 2024-09-13T17:45:40.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 0x7f94240379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f943c047360 con 0x7f944009d690 2024-09-13T17:45:40.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944495c640 1 -- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 msgr2=0x7f94240379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:40.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.158+0000 7f944495c640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 0x7f94240379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:40.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.166+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f943c01bba0 con 0x7f944009d690 2024-09-13T17:45:40.174 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.170+0000 7fbfc0896180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:45:40.248 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.246+0000 7fbfc0896180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:45:40.297 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.294+0000 7fbfc0896180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:45:40.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.318+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f944009bc00 con 0x7f944009d690 2024-09-13T17:45:40.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f9435ffb640 1 -- 172.21.15.96:0/3026807263 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+10555 (secure 0 0 0) 0x7f944009bc00 con 0x7f944009d690 2024-09-13T17:45:40.323 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:40.323 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":69,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:38.867051+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:40.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 msgr2=0x7f94240379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:40.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 0x7f94240379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 msgr2=0x7f94400a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f943c00c390 tx=0x7f943c00c850 comp rx=0 tx=0).stop 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 shutdown_connections 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f94240354e0 0x7f94240379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 --2- 172.21.15.96:0/3026807263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f944009d690 0x7f94400a6c30 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 >> 172.21.15.96:0/3026807263 conn(0x7f944006bc60 msgr2=0x7f9440139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 shutdown_connections 2024-09-13T17:45:40.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.322+0000 7f94473c1640 1 -- 172.21.15.96:0/3026807263 wait complete. 2024-09-13T17:45:40.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:45:25.156688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:45:25.809998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:40.345 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:40.442 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.438+0000 7fbfc0896180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:45:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31d23f640 1 Processor -- start 2024-09-13T17:45:40.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31d23f640 1 -- start start 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31d23f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa31809c930 0x7fa318098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31d23f640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31809aee0 con 0x7fa31809cd10 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31d23f640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31809be70 con 0x7fa31809c930 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa316d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa31809c930 0x7fa318098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa31809cd10 0x7fa318098fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44218/0 (socket says 172.21.15.96:44218) 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa31659c640 1 -- 172.21.15.96:0/2630750517 learned_addr learned my addr 172.21.15.96:0/2630750517 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa316d9d640 1 -- 172.21.15.96:0/2630750517 >> v1:172.21.15.96:6789/0 conn(0x7fa31809cd10 legacy=0x7fa318098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.466+0000 7fa316d9d640 1 -- 172.21.15.96:0/2630750517 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31806ae90 con 0x7fa31809c930 2024-09-13T17:45:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 --2- 172.21.15.96:0/2630750517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318098a60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009740 tx=0x7fa30c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99330ae0e754b0ba server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa315d9b640 1 -- 172.21.15.96:0/2630750517 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0ab530 con 0x7fa31809c930 2024-09-13T17:45:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa315d9b640 1 -- 172.21.15.96:0/2630750517 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0a7050 con 0x7fa31809c930 2024-09-13T17:45:40.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa315d9b640 1 -- 172.21.15.96:0/2630750517 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0ab7e0 con 0x7fa31809c930 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/2630750517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 msgr2=0x7fa318098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 --2- 172.21.15.96:0/2630750517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318098a60 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009740 tx=0x7fa30c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/2630750517 shutdown_connections 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 --2- 172.21.15.96:0/2630750517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318098a60 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/2630750517 >> 172.21.15.96:0/2630750517 conn(0x7fa31806e350 msgr2=0x7fa318070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/2630750517 shutdown_connections 2024-09-13T17:45:40.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/2630750517 wait complete. 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 Processor -- start 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- start start 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa318123010 con 0x7fa31809c930 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51528/0 (socket says 172.21.15.96:51528) 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 -- 172.21.15.96:0/3604455544 learned_addr learned my addr 172.21.15.96:0/3604455544 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 -- 172.21.15.96:0/3604455544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009420 con 0x7fa31809c930 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa316d9d640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa30c018fa0 tx=0x7fa30c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0b7660 con 0x7fa31809c930 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0b77c0 con 0x7fa31809c930 2024-09-13T17:45:40.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/3604455544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa318071a60 con 0x7fa31809c930 2024-09-13T17:45:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0b7aa0 con 0x7fa31809c930 2024-09-13T17:45:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.470+0000 7fa31d23f640 1 -- 172.21.15.96:0/3604455544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31811fd00 con 0x7fa31809c930 2024-09-13T17:45:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa31d23f640 1 -- 172.21.15.96:0/3604455544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa318123150 con 0x7fa31809c930 2024-09-13T17:45:40.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fa30c0b7c00 con 0x7fa31809c930 2024-09-13T17:45:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa2ff7fe640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 0x7fa3040379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fa30c0e2360 con 0x7fa31809c930 2024-09-13T17:45:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa31659c640 1 -- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 msgr2=0x7fa3040379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:40.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.474+0000 7fa31659c640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 0x7fa3040379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:40.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.478+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa30c0c7af0 con 0x7fa31809c930 2024-09-13T17:45:40.496 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.494+0000 7fbfc0896180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:45:40.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.674+0000 7fa31d23f640 1 -- 172.21.15.96:0/3604455544 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3181217c0 con 0x7fa31809c930 2024-09-13T17:45:40.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.678+0000 7fa2ff7fe640 1 -- 172.21.15.96:0/3604455544 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa30c0b00a0 con 0x7fa31809c930 2024-09-13T17:45:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.678+0000 7fa31659c640 1 -- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 msgr2=0x7fa3040379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:40.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.678+0000 7fa31659c640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 0x7fa3040379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-13T17:45:40.679 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:40.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 msgr2=0x7fa3040379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:40.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 0x7fa3040379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 msgr2=0x7fa318123a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fa30c018fa0 tx=0x7fa30c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 shutdown_connections 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fa3040354e0 0x7fa3040379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 --2- 172.21.15.96:0/3604455544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa31809c930 0x7fa318123a70 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 >> 172.21.15.96:0/3604455544 conn(0x7fa31806e350 msgr2=0x7fa3180a6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 shutdown_connections 2024-09-13T17:45:40.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.682+0000 7fa2fd7fa640 1 -- 172.21.15.96:0/3604455544 wait complete. 2024-09-13T17:45:40.703 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:40.716 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:45:40.717 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:45:40.717 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:45:40.724 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.722+0000 7fbfc0896180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:45:40.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d96400640 1 Processor -- start 2024-09-13T17:45:40.818 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d96400640 1 -- start start 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d96400640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3d901329b0 0x7f3d90132d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d96400640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d901327b0 con 0x7f3d9009dc30 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d96400640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9009bd90 con 0x7f3d901329b0 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.814+0000 7f3d8ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3d901329b0 0x7f3d90132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8f7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3d9009dc30 0x7f3d90135980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44226/0 (socket says 172.21.15.96:44226) 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8f7fe640 1 -- 172.21.15.96:0/2476294562 learned_addr learned my addr 172.21.15.96:0/2476294562 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:40.819 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8ffff640 1 -- 172.21.15.96:0/2476294562 >> v1:172.21.15.96:6789/0 conn(0x7f3d9009dc30 legacy=0x7f3d90135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8ffff640 1 -- 172.21.15.96:0/2476294562 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9009b2a0 con 0x7f3d901329b0 2024-09-13T17:45:40.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8ffff640 1 --2- 172.21.15.96:0/2476294562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901329b0 0x7f3d90132d90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009740 tx=0x7f3d8801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c553d9ecf2e98dd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8effd640 1 -- 172.21.15.96:0/2476294562 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d880ab530 con 0x7f3d901329b0 2024-09-13T17:45:40.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8effd640 1 -- 172.21.15.96:0/2476294562 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d880a7050 con 0x7f3d901329b0 2024-09-13T17:45:40.821 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d8effd640 1 -- 172.21.15.96:0/2476294562 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d880ab7e0 con 0x7f3d901329b0 2024-09-13T17:45:40.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 -- 172.21.15.96:0/2476294562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901329b0 msgr2=0x7f3d90132d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 --2- 172.21.15.96:0/2476294562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901329b0 0x7f3d90132d90 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009740 tx=0x7f3d8801c420 comp rx=0 tx=0).stop 2024-09-13T17:45:40.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 -- 172.21.15.96:0/2476294562 shutdown_connections 2024-09-13T17:45:40.822 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 --2- 172.21.15.96:0/2476294562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d901329b0 0x7f3d90132d90 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 -- 172.21.15.96:0/2476294562 >> 172.21.15.96:0/2476294562 conn(0x7f3d9006ef80 msgr2=0x7f3d900713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 -- 172.21.15.96:0/2476294562 shutdown_connections 2024-09-13T17:45:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.818+0000 7f3d96400640 1 -- 172.21.15.96:0/2476294562 wait complete. 2024-09-13T17:45:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 Processor -- start 2024-09-13T17:45:40.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 -- start start 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900a38d0 con 0x7f3d9009dc30 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51552/0 (socket says 172.21.15.96:51552) 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8ffff640 1 -- 172.21.15.96:0/4239125652 learned_addr learned my addr 172.21.15.96:0/4239125652 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:40.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8ffff640 1 -- 172.21.15.96:0/4239125652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88009420 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8ffff640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3d88019030 tx=0x7f3d8801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d880b7450 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d880b75d0 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d900a3440 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d880b78b0 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d900a3180 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f3d880b7a10 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d90135cf0 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 0x7f3d7c037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f3d880e2480 con 0x7f3d9009dc30 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8f7fe640 1 -- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 msgr2=0x7f3d7c037950 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:40.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.822+0000 7f3d8f7fe640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 0x7f3d7c037950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:40.829 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.826+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3d880c78d0 con 0x7f3d9009dc30 2024-09-13T17:45:40.861 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.858+0000 7fbfc0896180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:40.971 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:40.970+0000 7fbfc0896180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:40.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.974+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3d9009d340 con 0x7f3d9009dc30 2024-09-13T17:45:40.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.974+0000 7f3d8cff9640 1 -- 172.21.15.96:0/4239125652 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v69) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3d880bb020 con 0x7f3d9009dc30 2024-09-13T17:45:40.978 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:40.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 msgr2=0x7f3d7c037950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 0x7f3d7c037950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 msgr2=0x7f3d900a21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f3d88019030 tx=0x7f3d8801ac60 comp rx=0 tx=0).stop 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 shutdown_connections 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f3d7c035490 0x7f3d7c037950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 --2- 172.21.15.96:0/4239125652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d9009dc30 0x7f3d900a21d0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:40.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 >> 172.21.15.96:0/4239125652 conn(0x7f3d9006ef80 msgr2=0x7f3d901350b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:40.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 shutdown_connections 2024-09-13T17:45:40.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:40.978+0000 7f3d96400640 1 -- 172.21.15.96:0/4239125652 wait complete. 2024-09-13T17:45:40.995 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:41.035 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.034+0000 7fbfc0896180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:45:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc422d5640 1 Processor -- start 2024-09-13T17:45:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc422d5640 1 -- start start 2024-09-13T17:45:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc422d5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdc3c09d690 0x7fdc3c138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc422d5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c138050 con 0x7fdc3c1387d0 2024-09-13T17:45:41.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc422d5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c09c410 con 0x7fdc3c09d690 2024-09-13T17:45:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc3bfff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdc3c09d690 0x7fdc3c138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc3b7fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdc3c1387d0 0x7fdc3c134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44230/0 (socket says 172.21.15.96:44230) 2024-09-13T17:45:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.158+0000 7fdc3b7fe640 1 -- 172.21.15.96:0/1356844356 learned_addr learned my addr 172.21.15.96:0/1356844356 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3bfff640 1 -- 172.21.15.96:0/1356844356 >> v1:172.21.15.96:6789/0 conn(0x7fdc3c1387d0 legacy=0x7fdc3c134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3bfff640 1 -- 172.21.15.96:0/1356844356 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c09cba0 con 0x7fdc3c09d690 2024-09-13T17:45:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3bfff640 1 --2- 172.21.15.96:0/1356844356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c138290 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009d90 tx=0x7fdc3401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daabbf3d18853112 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3affd640 1 -- 172.21.15.96:0/1356844356 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc340a8e10 con 0x7fdc3c09d690 2024-09-13T17:45:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3affd640 1 -- 172.21.15.96:0/1356844356 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340b0440 con 0x7fdc3c09d690 2024-09-13T17:45:41.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc3affd640 1 -- 172.21.15.96:0/1356844356 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc340b0700 con 0x7fdc3c09d690 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 -- 172.21.15.96:0/1356844356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 msgr2=0x7fdc3c138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 --2- 172.21.15.96:0/1356844356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c138290 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fdc34009d90 tx=0x7fdc3401f150 comp rx=0 tx=0).stop 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 -- 172.21.15.96:0/1356844356 shutdown_connections 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 --2- 172.21.15.96:0/1356844356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c138290 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 -- 172.21.15.96:0/1356844356 >> 172.21.15.96:0/1356844356 conn(0x7fdc3c06bc60 msgr2=0x7fdc3c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 -- 172.21.15.96:0/1356844356 shutdown_connections 2024-09-13T17:45:41.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 -- 172.21.15.96:0/1356844356 wait complete. 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.162+0000 7fdc422d5640 1 Processor -- start 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 -- start start 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0a4d00 con 0x7fdc3c09d690 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51570/0 (socket says 172.21.15.96:51570) 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3bfff640 1 -- 172.21.15.96:0/2208229612 learned_addr learned my addr 172.21.15.96:0/2208229612 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3bfff640 1 -- 172.21.15.96:0/2208229612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34009a70 con 0x7fdc3c09d690 2024-09-13T17:45:41.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3bfff640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdc3401bd60 tx=0x7fdc3401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc340ba540 con 0x7fdc3c09d690 2024-09-13T17:45:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340ba6a0 con 0x7fdc3c09d690 2024-09-13T17:45:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c0a4f30 con 0x7fdc3c09d690 2024-09-13T17:45:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdc340ba980 con 0x7fdc3c09d690 2024-09-13T17:45:41.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c0a6f00 con 0x7fdc3c09d690 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c0a4420 con 0x7fdc3c09d690 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fdc340b7050 con 0x7fdc3c09d690 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 0x7fdc280379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fdc340e6500 con 0x7fdc3c09d690 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3b7fe640 1 -- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 msgr2=0x7fdc280379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:41.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.166+0000 7fdc3b7fe640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 0x7fdc280379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:41.171 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.170+0000 7fbfc0896180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:41.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.170+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdc340bae60 con 0x7fdc3c09d690 2024-09-13T17:45:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.330+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc3c09da70 con 0x7fdc3c09d690 2024-09-13T17:45:41.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.330+0000 7fdc38ff9640 1 -- 172.21.15.96:0/2208229612 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc340b5020 con 0x7fdc3c09d690 2024-09-13T17:45:41.333 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 msgr2=0x7fdc280379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 0x7fdc280379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 msgr2=0x7fdc3c0a69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fdc3401bd60 tx=0x7fdc3401bef0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 shutdown_connections 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fdc280354e0 0x7fdc280379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 --2- 172.21.15.96:0/2208229612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc3c09d690 0x7fdc3c0a69c0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 >> 172.21.15.96:0/2208229612 conn(0x7fdc3c06bc60 msgr2=0x7fdc3c0993a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 shutdown_connections 2024-09-13T17:45:41.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.334+0000 7fdc422d5640 1 -- 172.21.15.96:0/2208229612 wait complete. 2024-09-13T17:45:41.352 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:45:41.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.474+0000 7fc1ffb15640 1 Processor -- start 2024-09-13T17:45:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.474+0000 7fc1ffb15640 1 -- start start 2024-09-13T17:45:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1ffb15640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc1f813af50 0x7fc1f813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1ffb15640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f8138050 con 0x7fc1f809d690 2024-09-13T17:45:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1ffb15640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f809c410 con 0x7fc1f813af50 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fd0b0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc1f813af50 0x7fc1f813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fd8b1640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc1f809d690 0x7fc1f8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44232/0 (socket says 172.21.15.96:44232) 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fd8b1640 1 -- 172.21.15.96:0/3643162045 learned_addr learned my addr 172.21.15.96:0/3643162045 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fd0b0640 1 -- 172.21.15.96:0/3643162045 >> v1:172.21.15.96:6789/0 conn(0x7fc1f809d690 legacy=0x7fc1f8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fd0b0640 1 -- 172.21.15.96:0/3643162045 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f809cba0 con 0x7fc1f813af50 2024-09-13T17:45:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.478+0000 7fc1fc8af640 1 -- 172.21.15.96:0/3643162045 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955764315 0 0) 0x7fc1e8001120 con 0x7fc1f809d690 2024-09-13T17:45:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd0b0640 1 --2- 172.21.15.96:0/3643162045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f813af50 0x7fc1f813d340 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009740 tx=0x7fc1f401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0f6b6fb672cf91 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fc8af640 1 -- 172.21.15.96:0/3643162045 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f40a8840 con 0x7fc1f813af50 2024-09-13T17:45:41.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fc8af640 1 -- 172.21.15.96:0/3643162045 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f40a89a0 con 0x7fc1f813af50 2024-09-13T17:45:41.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fc8af640 1 -- 172.21.15.96:0/3643162045 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1f40a8c60 con 0x7fc1f813af50 2024-09-13T17:45:41.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3643162045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f813af50 msgr2=0x7fc1f813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3643162045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f813af50 0x7fc1f813d340 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fc1f4009740 tx=0x7fc1f401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:41.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3643162045 shutdown_connections 2024-09-13T17:45:41.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3643162045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f813af50 0x7fc1f813d340 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3643162045 >> 172.21.15.96:0/3643162045 conn(0x7fc1f806bc60 msgr2=0x7fc1f806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:41.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3643162045 shutdown_connections 2024-09-13T17:45:41.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3643162045 wait complete. 2024-09-13T17:45:41.485 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 Processor -- start 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- start start 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80a5e20 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd8b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd8b1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51592/0 (socket says 172.21.15.96:51592) 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd8b1640 1 -- 172.21.15.96:0/3862291983 learned_addr learned my addr 172.21.15.96:0/3862291983 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd8b1640 1 -- 172.21.15.96:0/3862291983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f4009420 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1fd8b1640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c390 tx=0x7fc1e800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e8007410 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8007590 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f80a27e0 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e80047f0 con 0x7fc1f809d690 2024-09-13T17:45:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f80a4250 con 0x7fc1f809d690 2024-09-13T17:45:41.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fc1e8004950 con 0x7fc1f809d690 2024-09-13T17:45:41.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f809bc00 con 0x7fc1f809d690 2024-09-13T17:45:41.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 0x7fc1dc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.482+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7fc1e8047420 con 0x7fc1f809d690 2024-09-13T17:45:41.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.486+0000 7fc1fd0b0640 1 -- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 msgr2=0x7fc1dc0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:41.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.486+0000 7fc1fd0b0640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 0x7fc1dc0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:41.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.490+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fc1f809bc00 con 0x7fc1f809d690 2024-09-13T17:45:41.650 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.646+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc1f8138050 con 0x7fc1f809d690 2024-09-13T17:45:41.651 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ee7fc640 1 -- 172.21.15.96:0/3862291983 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 1 entries v69) v1 ==== 80+0+58 (secure 0 0 0) 0x7fc1f8138050 con 0x7fc1f809d690 2024-09-13T17:45:41.651 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1206016791 2024-09-13T18:45:38.078633+0000 2024-09-13T17:45:41.651 INFO:teuthology.orchestra.run.smithi096.stderr:listed 1 entries 2024-09-13T17:45:41.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 msgr2=0x7fc1dc0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:41.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 0x7fc1dc0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 msgr2=0x7fc1f80a67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fc1e800c390 tx=0x7fc1e800c850 comp rx=0 tx=0).stop 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 shutdown_connections 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7fc1dc0354e0 0x7fc1dc0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 --2- 172.21.15.96:0/3862291983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1f809d690 0x7fc1f80a67f0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 >> 172.21.15.96:0/3862291983 conn(0x7fc1f806bc60 msgr2=0x7fc1f8139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 shutdown_connections 2024-09-13T17:45:41.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.650+0000 7fc1ffb15640 1 -- 172.21.15.96:0/3862291983 wait complete. 2024-09-13T17:45:41.661 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.658+0000 7fbfc0896180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:45:41.669 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:45:41.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:41.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-13T17:45:41.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi096.stderr:ceph-fuse[44241]: fuse finished with error 0 and tester_r 0 2024-09-13T17:45:41.680 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:45:41.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:41.680 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-13T17:45:41.739 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.738+0000 7fbfc0896180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:45:41.742 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:41.742 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:41.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:41.884 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.882+0000 7fbfc0896180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:45:41.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.950+0000 7f884ee1b640 1 Processor -- start 2024-09-13T17:45:41.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.950+0000 7f884ee1b640 1 -- start start 2024-09-13T17:45:41.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.950+0000 7f884ee1b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f88480a4900 0x7f884809e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.950+0000 7f884ee1b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8848071740 con 0x7f88480a4ce0 2024-09-13T17:45:41.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.950+0000 7f884ee1b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884809e250 con 0x7f88480a4900 2024-09-13T17:45:41.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f88480a4900 0x7f884809e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f88480a4900 0x7f884809e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51598/0 (socket says 172.21.15.96:51598) 2024-09-13T17:45:41.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 -- 172.21.15.96:0/3696050738 learned_addr learned my addr 172.21.15.96:0/3696050738 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:41.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 -- 172.21.15.96:0/3696050738 >> v1:172.21.15.96:6789/0 conn(0x7f88480a4ce0 legacy=0x7f88480a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 -- 172.21.15.96:0/3696050738 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88480a5c70 con 0x7f88480a4900 2024-09-13T17:45:41.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884cbb7640 1 --2- 172.21.15.96:0/3696050738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884809e9a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f88340097f0 tx=0x7f883401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38e9c1e61208848 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f88477fe640 1 -- 172.21.15.96:0/3696050738 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88340ab530 con 0x7f88480a4900 2024-09-13T17:45:41.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f88477fe640 1 -- 172.21.15.96:0/3696050738 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88340a7050 con 0x7f88480a4900 2024-09-13T17:45:41.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f88477fe640 1 -- 172.21.15.96:0/3696050738 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88340ab7e0 con 0x7f88480a4900 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 -- 172.21.15.96:0/3696050738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 msgr2=0x7f884809e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3696050738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884809e9a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f88340097f0 tx=0x7f883401c450 comp rx=0 tx=0).stop 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 -- 172.21.15.96:0/3696050738 shutdown_connections 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3696050738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884809e9a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 -- 172.21.15.96:0/3696050738 >> 172.21.15.96:0/3696050738 conn(0x7f884806a150 msgr2=0x7f884806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 -- 172.21.15.96:0/3696050738 shutdown_connections 2024-09-13T17:45:41.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.954+0000 7f884ee1b640 1 -- 172.21.15.96:0/3696050738 wait complete. 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 Processor -- start 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 -- start start 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884811db50 con 0x7f88480a4900 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884cbb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:41.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884cbb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51602/0 (socket says 172.21.15.96:51602) 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884cbb7640 1 -- 172.21.15.96:0/3058063529 learned_addr learned my addr 172.21.15.96:0/3058063529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884cbb7640 1 -- 172.21.15.96:0/3058063529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88340094d0 con 0x7f88480a4900 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884cbb7640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f883401a080 tx=0x7f88340191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88340b4440 con 0x7f88480a4900 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88340b45a0 con 0x7f88480a4900 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88480706d0 con 0x7f88480a4900 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f88340b4860 con 0x7f88480a4900 2024-09-13T17:45:41.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884811e200 con 0x7f88480a4900 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f88340b2070 con 0x7f88480a4900 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8848073570 con 0x7f88480a4900 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 0x7f882c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f8847fff640 1 -- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 msgr2=0x7f882c0379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f88340e1790 con 0x7f88480a4900 2024-09-13T17:45:41.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.958+0000 7f8847fff640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 0x7f882c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:41.963 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:41.958+0000 7fbfc0896180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:45:41.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:41.962+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f88340c46c0 con 0x7f88480a4900 2024-09-13T17:45:42.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.122+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8848070980 con 0x7f88480a4900 2024-09-13T17:45:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f88457fa640 1 -- 172.21.15.96:0/3058063529 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+10555 (secure 0 0 0) 0x7f88340b8090 con 0x7f88480a4900 2024-09-13T17:45:42.127 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:42.127 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":69,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:38.867051+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 msgr2=0x7f882c0379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 0x7f882c0379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 msgr2=0x7f884811e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f883401a080 tx=0x7f88340191f0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 shutdown_connections 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f882c0354e0 0x7f882c0379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 --2- 172.21.15.96:0/3058063529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88480a4900 0x7f884811e5b0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 >> 172.21.15.96:0/3058063529 conn(0x7f884806a150 msgr2=0x7f8848138960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 shutdown_connections 2024-09-13T17:45:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.126+0000 7f884ee1b640 1 -- 172.21.15.96:0/3058063529 wait complete. 2024-09-13T17:45:42.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:45:25.156688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:45:25.809998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:42.146 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:42.252 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:42.250+0000 7fbfc0896180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:45:42.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.258+0000 7f859cb18640 1 Processor -- start 2024-09-13T17:45:42.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.258+0000 7f859cb18640 1 -- start start 2024-09-13T17:45:42.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f859cb18640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f85980a53f0 0x7f859809e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f859cb18640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8598071740 con 0x7f85980a4900 2024-09-13T17:45:42.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f859cb18640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859809e250 con 0x7f85980a53f0 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f859659c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f85980a4900 0x7f85980a4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:44252/0 (socket says 172.21.15.96:44252) 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f859659c640 1 -- 172.21.15.96:0/3783688654 learned_addr learned my addr 172.21.15.96:0/3783688654 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f858ffff640 1 --2- 172.21.15.96:0/3783688654 >> v2:172.21.15.96:3300/0 conn(0x7f85980a53f0 0x7f859809e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f8595d9b640 1 -- 172.21.15.96:0/3783688654 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624798625 0 0) 0x7f8598071740 con 0x7f85980a4900 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f8595d9b640 1 -- 172.21.15.96:0/3783688654 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8584003530 con 0x7f85980a4900 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f858ffff640 1 -- 172.21.15.96:0/3783688654 >> v1:172.21.15.96:6789/0 conn(0x7f85980a4900 legacy=0x7f85980a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f858ffff640 1 -- 172.21.15.96:0/3783688654 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85980a5c70 con 0x7f85980a53f0 2024-09-13T17:45:42.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f858ffff640 1 --2- 172.21.15.96:0/3783688654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a53f0 0x7f859809e9a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8580009740 tx=0x7f858001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c508b7baea8ce601 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f8595d9b640 1 -- 172.21.15.96:0/3783688654 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85800ab530 con 0x7f85980a53f0 2024-09-13T17:45:42.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f8595d9b640 1 -- 172.21.15.96:0/3783688654 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85800a7050 con 0x7f85980a53f0 2024-09-13T17:45:42.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.262+0000 7f8595d9b640 1 -- 172.21.15.96:0/3783688654 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f85800ab7c0 con 0x7f85980a53f0 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/3783688654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a53f0 msgr2=0x7f859809e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 --2- 172.21.15.96:0/3783688654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a53f0 0x7f859809e9a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f8580009740 tx=0x7f858001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/3783688654 shutdown_connections 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 --2- 172.21.15.96:0/3783688654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a53f0 0x7f859809e9a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/3783688654 >> 172.21.15.96:0/3783688654 conn(0x7f859806a150 msgr2=0x7f859806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/3783688654 shutdown_connections 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/3783688654 wait complete. 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 Processor -- start 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- start start 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598071580 con 0x7f85980a4900 2024-09-13T17:45:42.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51618/0 (socket says 172.21.15.96:51618) 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859659c640 1 -- 172.21.15.96:0/918780359 learned_addr learned my addr 172.21.15.96:0/918780359 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859659c640 1 -- 172.21.15.96:0/918780359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8580009420 con 0x7f85980a4900 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859659c640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f858800c340 tx=0x7f858800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8588005c20 con 0x7f85980a4900 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8588005d80 con 0x7f85980a4900 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859813c530 con 0x7f85980a4900 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8588007220 con 0x7f85980a4900 2024-09-13T17:45:42.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859811e920 con 0x7f85980a4900 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.266+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 34) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f8588007380 con 0x7f85980a4900 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.270+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8598073570 con 0x7f85980a4900 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.270+0000 7f858effd640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 0x7f85780379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.270+0000 7f858ffff640 1 -- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 msgr2=0x7f85780379a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6824/3517445048 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.270+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 3482+0+0 (secure 0 0 0) 0x7f8588046c10 con 0x7f85980a4900 2024-09-13T17:45:42.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.270+0000 7f858ffff640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 0x7f85780379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-13T17:45:42.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.278+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f858801b3b0 con 0x7f85980a4900 2024-09-13T17:45:42.312 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:42.310+0000 7fbfc0896180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:45:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.338+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f8588010c10 con 0x7f85980a4900 2024-09-13T17:45:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.338+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 msgr2=0x7f85780379a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-13T17:45:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.338+0000 7f858effd640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 0x7f85780379a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.434+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f859813c530 con 0x7f85980a4900 2024-09-13T17:45:42.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.434+0000 7f858effd640 1 -- 172.21.15.96:0/918780359 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f859813c530 con 0x7f85980a4900 2024-09-13T17:45:42.438 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 msgr2=0x7f85981cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f858800c340 tx=0x7f858800c800 comp rx=0 tx=0).stop 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 shutdown_connections 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:6824/3517445048,v1:172.21.15.96:6825/3517445048] conn(0x7f85780354e0 0x7f85780379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 --2- 172.21.15.96:0/918780359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f85980a4900 0x7f85981cc1a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 >> 172.21.15.96:0/918780359 conn(0x7f859806a150 msgr2=0x7f8598138630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 shutdown_connections 2024-09-13T17:45:42.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.438+0000 7f859cb18640 1 -- 172.21.15.96:0/918780359 wait complete. 2024-09-13T17:45:42.455 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:42.454+0000 7fe8513d1180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T17:45:42.456 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:42.566 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 Processor -- start 2024-09-13T17:45:42.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- start start 2024-09-13T17:45:42.567 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e1009c930 0x7f7e10098a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e1009aee0 con 0x7f7e1009cd10 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e1009be70 con 0x7f7e1009c930 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e1009c930 0x7f7e10098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f7e1009c930 0x7f7e10098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39654/0 (socket says 172.21.15.96:39654) 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 -- 172.21.15.96:0/1147825157 learned_addr learned my addr 172.21.15.96:0/1147825157 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e0ffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f7e1009cd10 0x7f7e10098fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55556/0 (socket says 172.21.15.96:55556) 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 -- 172.21.15.96:0/1147825157 >> v1:172.21.15.96:6789/0 conn(0x7f7e1009cd10 legacy=0x7f7e10098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 -- 172.21.15.96:0/1147825157 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e1006ae90 con 0x7f7e1009c930 2024-09-13T17:45:42.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e14ce5640 1 --2- 172.21.15.96:0/1147825157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10098a60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097f0 tx=0x7f7e0401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4af5a2675d8a39 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e0f7fe640 1 -- 172.21.15.96:0/1147825157 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e040ab530 con 0x7f7e1009c930 2024-09-13T17:45:42.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e0f7fe640 1 -- 172.21.15.96:0/1147825157 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040a7050 con 0x7f7e1009c930 2024-09-13T17:45:42.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e0f7fe640 1 -- 172.21.15.96:0/1147825157 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e040ab7e0 con 0x7f7e1009c930 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- 172.21.15.96:0/1147825157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 msgr2=0x7f7e10098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 --2- 172.21.15.96:0/1147825157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10098a60 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097f0 tx=0x7f7e0401c450 comp rx=0 tx=0).stop 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- 172.21.15.96:0/1147825157 shutdown_connections 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 --2- 172.21.15.96:0/1147825157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10098a60 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.566+0000 7f7e16f49640 1 -- 172.21.15.96:0/1147825157 >> 172.21.15.96:0/1147825157 conn(0x7f7e1006e350 msgr2=0x7f7e10070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- 172.21.15.96:0/1147825157 shutdown_connections 2024-09-13T17:45:42.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- 172.21.15.96:0/1147825157 wait complete. 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 Processor -- start 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- start start 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10121620 con 0x7f7e1009c930 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e14ce5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e14ce5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39656/0 (socket says 172.21.15.96:39656) 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e14ce5640 1 -- 172.21.15.96:0/3268365152 learned_addr learned my addr 172.21.15.96:0/3268365152 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:42.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e14ce5640 1 -- 172.21.15.96:0/3268365152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040094d0 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e14ce5640 1 --2- 172.21.15.96:0/3268365152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7e040ae040 tx=0x7f7e0401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e040b6630 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e040b6790 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e10121f00 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e040b6a30 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e101212d0 con 0x7f7e1009c930 2024-09-13T17:45:42.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f7e040b2040 con 0x7f7e1009c930 2024-09-13T17:45:42.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e10122e70 con 0x7f7e1009c930 2024-09-13T17:45:42.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.570+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f7e040e1ac0 con 0x7f7e1009c930 2024-09-13T17:45:42.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.574+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f7e040bd720 con 0x7f7e1009c930 2024-09-13T17:45:42.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.722+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e10098a60 con 0x7f7e1009c930 2024-09-13T17:45:42.726 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.722+0000 7f7e0d7fa640 1 -- 172.21.15.96:0/3268365152 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e040b0070 con 0x7f7e1009c930 2024-09-13T17:45:42.726 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:42.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 msgr2=0x7f7e10123bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 --2- 172.21.15.96:0/3268365152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7e040ae040 tx=0x7f7e0401ac90 comp rx=0 tx=0).stop 2024-09-13T17:45:42.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 shutdown_connections 2024-09-13T17:45:42.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 --2- 172.21.15.96:0/3268365152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e1009c930 0x7f7e10123bc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 >> 172.21.15.96:0/3268365152 conn(0x7f7e1006e350 msgr2=0x7f7e1013af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 shutdown_connections 2024-09-13T17:45:42.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.726+0000 7f7e16f49640 1 -- 172.21.15.96:0/3268365152 wait complete. 2024-09-13T17:45:42.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:42.843 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:42.842+0000 7fe8513d1180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T17:45:42.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa919091640 1 Processor -- start 2024-09-13T17:45:42.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa919091640 1 -- start start 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa919091640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa91409d690 0x7fa914138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa919091640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa914138050 con 0x7fa9141387d0 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa919091640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa91409c410 con 0x7fa91409d690 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa912d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa91409d690 0x7fa914138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa91259c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fa9141387d0 0x7fa914134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55570/0 (socket says 172.21.15.96:55570) 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa912d9d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa91409d690 0x7fa914138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39670/0 (socket says 172.21.15.96:39670) 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.866+0000 7fa91259c640 1 -- 172.21.15.96:0/3315873160 learned_addr learned my addr 172.21.15.96:0/3315873160 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa912d9d640 1 -- 172.21.15.96:0/3315873160 >> v1:172.21.15.96:6789/0 conn(0x7fa9141387d0 legacy=0x7fa914134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa912d9d640 1 -- 172.21.15.96:0/3315873160 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91409cba0 con 0x7fa91409d690 2024-09-13T17:45:42.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa912d9d640 1 --2- 172.21.15.96:0/3315873160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa914138290 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa904009740 tx=0x7fa90401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c680d7bf51d98e server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa911d9b640 1 -- 172.21.15.96:0/3315873160 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9040a8840 con 0x7fa91409d690 2024-09-13T17:45:42.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa911d9b640 1 -- 172.21.15.96:0/3315873160 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9040a89a0 con 0x7fa91409d690 2024-09-13T17:45:42.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa911d9b640 1 -- 172.21.15.96:0/3315873160 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9040a8c60 con 0x7fa91409d690 2024-09-13T17:45:42.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 -- 172.21.15.96:0/3315873160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 msgr2=0x7fa914138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 --2- 172.21.15.96:0/3315873160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa914138290 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fa904009740 tx=0x7fa90401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 -- 172.21.15.96:0/3315873160 shutdown_connections 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 --2- 172.21.15.96:0/3315873160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa914138290 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 -- 172.21.15.96:0/3315873160 >> 172.21.15.96:0/3315873160 conn(0x7fa91406bc60 msgr2=0x7fa91406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 -- 172.21.15.96:0/3315873160 shutdown_connections 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.870+0000 7fa919091640 1 -- 172.21.15.96:0/3315873160 wait complete. 2024-09-13T17:45:42.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 Processor -- start 2024-09-13T17:45:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 -- start start 2024-09-13T17:45:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9140a5e10 con 0x7fa91409d690 2024-09-13T17:45:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa912d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:42.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa912d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39672/0 (socket says 172.21.15.96:39672) 2024-09-13T17:45:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa912d9d640 1 -- 172.21.15.96:0/4050220802 learned_addr learned my addr 172.21.15.96:0/4050220802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:42.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa912d9d640 1 -- 172.21.15.96:0/4050220802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa904009420 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa912d9d640 1 --2- 172.21.15.96:0/4050220802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa90401c840 tx=0x7fa90401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9040b0840 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9040b09a0 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9140a70a0 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9040b0c60 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9140a4300 con 0x7fa91409d690 2024-09-13T17:45:42.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7fa9040b7040 con 0x7fa91409d690 2024-09-13T17:45:42.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91409bc00 con 0x7fa91409d690 2024-09-13T17:45:42.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.874+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7fa9040e4fc0 con 0x7fa91409d690 2024-09-13T17:45:42.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:42.878+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa91409bc00 con 0x7fa91409d690 2024-09-13T17:45:42.980 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:42.978+0000 7fe8513d1180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T17:45:43.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.026+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa91409bc00 con 0x7fa91409d690 2024-09-13T17:45:43.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.026+0000 7fa8fb7fe640 1 -- 172.21.15.96:0/4050220802 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa91409bc00 con 0x7fa91409d690 2024-09-13T17:45:43.030 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 msgr2=0x7fa9140a6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 --2- 172.21.15.96:0/4050220802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fa90401c840 tx=0x7fa90401b9b0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 shutdown_connections 2024-09-13T17:45:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 --2- 172.21.15.96:0/4050220802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa91409d690 0x7fa9140a6b60 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 >> 172.21.15.96:0/4050220802 conn(0x7fa91406bc60 msgr2=0x7fa9141bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:43.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 shutdown_connections 2024-09-13T17:45:43.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.030+0000 7fa919091640 1 -- 172.21.15.96:0/4050220802 wait complete. 2024-09-13T17:45:43.050 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-13T17:45:43.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.154+0000 7f10f8c15640 1 Processor -- start 2024-09-13T17:45:43.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.154+0000 7f10f8c15640 1 -- start start 2024-09-13T17:45:43.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f8c15640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f10f409d690 0x7f10f4138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:43.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f8c15640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f4138050 con 0x7f10f41387d0 2024-09-13T17:45:43.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f8c15640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f409c410 con 0x7f10f409d690 2024-09-13T17:45:43.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f259c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f10f409d690 0x7f10f4138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:43.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f1d9b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f10f41387d0 0x7f10f4134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55582/0 (socket says 172.21.15.96:55582) 2024-09-13T17:45:43.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f1d9b640 1 -- 172.21.15.96:0/24135285 learned_addr learned my addr 172.21.15.96:0/24135285 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:43.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f259c640 1 -- 172.21.15.96:0/24135285 >> v1:172.21.15.96:6789/0 conn(0x7f10f41387d0 legacy=0x7f10f4134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f159a640 1 -- 172.21.15.96:0/24135285 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246537591 0 0) 0x7f10e8001120 con 0x7f10f41387d0 2024-09-13T17:45:43.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f259c640 1 -- 172.21.15.96:0/24135285 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f409cba0 con 0x7f10f409d690 2024-09-13T17:45:43.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f259c640 1 --2- 172.21.15.96:0/24135285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f4138290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009740 tx=0x7f10e401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9644731276ae2087 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:43.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f159a640 1 -- 172.21.15.96:0/24135285 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e40a8840 con 0x7f10f409d690 2024-09-13T17:45:43.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f159a640 1 -- 172.21.15.96:0/24135285 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40a89a0 con 0x7f10f409d690 2024-09-13T17:45:43.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.158+0000 7f10f159a640 1 -- 172.21.15.96:0/24135285 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e40a8c60 con 0x7f10f409d690 2024-09-13T17:45:43.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/24135285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 msgr2=0x7f10f4138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 --2- 172.21.15.96:0/24135285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f4138290 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f10e4009740 tx=0x7f10e401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:43.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/24135285 shutdown_connections 2024-09-13T17:45:43.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 --2- 172.21.15.96:0/24135285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f4138290 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/24135285 >> 172.21.15.96:0/24135285 conn(0x7f10f406bc60 msgr2=0x7f10f406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/24135285 shutdown_connections 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/24135285 wait complete. 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 Processor -- start 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- start start 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f40a2840 con 0x7f10f409d690 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39688/0 (socket says 172.21.15.96:39688) 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f259c640 1 -- 172.21.15.96:0/92532142 learned_addr learned my addr 172.21.15.96:0/92532142 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f259c640 1 -- 172.21.15.96:0/92532142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4009420 con 0x7f10f409d690 2024-09-13T17:45:43.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f259c640 1 --2- 172.21.15.96:0/92532142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f10e401b820 tx=0x7f10e401d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e4004150 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e40042b0 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f40a5e90 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f10e40bc720 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f40a6150 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f10e40bc880 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f409bc00 con 0x7f10f409d690 2024-09-13T17:45:43.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.162+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f10e40e5530 con 0x7f10f409d690 2024-09-13T17:45:43.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.170+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f10f409bc00 con 0x7f10f409d690 2024-09-13T17:45:43.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.330+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10f409bc00 con 0x7f10f409d690 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.330+0000 7f10e2ffd640 1 -- 172.21.15.96:0/92532142 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v70) v1 ==== 80+0+469 (secure 0 0 0) 0x7f10f409bc00 con 0x7f10f409d690 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2416365020 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/3625984202 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6825/3517445048 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6824/3517445048 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1018463101 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.333 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/928338259 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.334 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1348634400 2024-09-14T17:45:42.332380+0000 2024-09-13T17:45:43.334 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1206016791 2024-09-13T18:45:38.078633+0000 2024-09-13T17:45:43.334 INFO:teuthology.orchestra.run.smithi096.stderr:listed 8 entries 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 msgr2=0x7f10f40a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 --2- 172.21.15.96:0/92532142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f10e401b820 tx=0x7f10e401d450 comp rx=0 tx=0).stop 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 shutdown_connections 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 --2- 172.21.15.96:0/92532142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10f409d690 0x7f10f40a68d0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 >> 172.21.15.96:0/92532142 conn(0x7f10f406bc60 msgr2=0x7f10f409e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 shutdown_connections 2024-09-13T17:45:43.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.334+0000 7f10f8c15640 1 -- 172.21.15.96:0/92532142 wait complete. 2024-09-13T17:45:43.352 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi096.front.sepia.ceph.com... 2024-09-13T17:45:43.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:43.353 DEBUG:teuthology.orchestra.run.smithi096:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-13T17:45:43.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi096.stderr:ceph-fuse[44348]: fuse finished with error 0 and tester_r 0 2024-09-13T17:45:43.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:45:43.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:43.365 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-13T17:45:43.412 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.410+0000 7fe8513d1180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:43.463 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.462+0000 7fe8513d1180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T17:45:43.511 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.510+0000 7fe8513d1180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T17:45:43.532 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:43.587 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.586+0000 7fe8513d1180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T17:45:43.637 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.634+0000 7fe8513d1180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T17:45:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a4e605640 1 Processor -- start 2024-09-13T17:45:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a4e605640 1 -- start start 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a4e605640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6a4809d7e0 0x7f6a4813af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a4e605640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a4809d090 con 0x7f6a48136100 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a4e605640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a4809cd80 con 0x7f6a4809d7e0 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a477fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6a4809d7e0 0x7f6a4813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a47fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6a48136100 0x7f6a481364e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55588/0 (socket says 172.21.15.96:55588) 2024-09-13T17:45:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a47fff640 1 -- 172.21.15.96:0/2295787876 learned_addr learned my addr 172.21.15.96:0/2295787876 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a477fe640 1 -- 172.21.15.96:0/2295787876 >> v1:172.21.15.96:6789/0 conn(0x7f6a48136100 legacy=0x7f6a481364e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a46ffd640 1 -- 172.21.15.96:0/2295787876 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838242864 0 0) 0x7f6a4809d090 con 0x7f6a48136100 2024-09-13T17:45:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.690+0000 7f6a477fe640 1 -- 172.21.15.96:0/2295787876 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4809dec0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a477fe640 1 --2- 172.21.15.96:0/2295787876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a4813af80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009740 tx=0x7f6a3c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352749fc7838524f server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a46ffd640 1 -- 172.21.15.96:0/2295787876 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c0ab530 con 0x7f6a4809d7e0 2024-09-13T17:45:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a46ffd640 1 -- 172.21.15.96:0/2295787876 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0a7050 con 0x7f6a4809d7e0 2024-09-13T17:45:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a46ffd640 1 -- 172.21.15.96:0/2295787876 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a3c0ab7e0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- 172.21.15.96:0/2295787876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 msgr2=0x7f6a4813af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 --2- 172.21.15.96:0/2295787876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a4813af80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009740 tx=0x7f6a3c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- 172.21.15.96:0/2295787876 shutdown_connections 2024-09-13T17:45:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 --2- 172.21.15.96:0/2295787876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a4813af80 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- 172.21.15.96:0/2295787876 >> 172.21.15.96:0/2295787876 conn(0x7f6a4806f270 msgr2=0x7f6a48071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:43.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- 172.21.15.96:0/2295787876 shutdown_connections 2024-09-13T17:45:43.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- 172.21.15.96:0/2295787876 wait complete. 2024-09-13T17:45:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 Processor -- start 2024-09-13T17:45:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.694+0000 7f6a4e605640 1 -- start start 2024-09-13T17:45:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a4e605640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a4e605640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a480a13a0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39708/0 (socket says 172.21.15.96:39708) 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a47fff640 1 -- 172.21.15.96:0/1008490594 learned_addr learned my addr 172.21.15.96:0/1008490594 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a47fff640 1 -- 172.21.15.96:0/1008490594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c009420 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a47fff640 1 --2- 172.21.15.96:0/1008490594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c450 tx=0x7f6a3800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a38005c20 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38005d80 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a480a2bf0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6a380071a0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a480a3bc0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f6a38007410 con 0x7f6a4809d7e0 2024-09-13T17:45:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a480a3e10 con 0x7f6a4809d7e0 2024-09-13T17:45:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.698+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f6a38046ec0 con 0x7f6a4809d7e0 2024-09-13T17:45:43.705 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.702+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6a480a3e10 con 0x7f6a4809d7e0 2024-09-13T17:45:43.713 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.710+0000 7fe8513d1180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T17:45:43.763 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.762+0000 7fe8513d1180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T17:45:43.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.882+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a480a3e10 con 0x7f6a4809d7e0 2024-09-13T17:45:43.889 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.886+0000 7f6a44ff9640 1 -- 172.21.15.96:0/1008490594 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f6a480a3e10 con 0x7f6a4809d7e0 2024-09-13T17:45:43.889 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:43.889 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:43.889 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 msgr2=0x7f6a480a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 --2- 172.21.15.96:0/1008490594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c450 tx=0x7f6a3800c910 comp rx=0 tx=0).stop 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 shutdown_connections 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 --2- 172.21.15.96:0/1008490594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a4809d7e0 0x7f6a480a1d60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 >> 172.21.15.96:0/1008490594 conn(0x7f6a4806f270 msgr2=0x7f6a4806f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 shutdown_connections 2024-09-13T17:45:43.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:43.890+0000 7f6a4e605640 1 -- 172.21.15.96:0/1008490594 wait complete. 2024-09-13T17:45:43.904 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.902+0000 7fe8513d1180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T17:45:43.906 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:43.956 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:43.954+0000 7fe8513d1180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T17:45:44.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.054+0000 7f87de951640 1 Processor -- start 2024-09-13T17:45:44.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.054+0000 7f87de951640 1 -- start start 2024-09-13T17:45:44.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87de951640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87d813af50 0x7f87d813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87de951640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d8138050 con 0x7f87d809d690 2024-09-13T17:45:44.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87de951640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d809c410 con 0x7f87d813af50 2024-09-13T17:45:44.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f87d813af50 0x7f87d813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d7fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f87d809d690 0x7f87d8138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55592/0 (socket says 172.21.15.96:55592) 2024-09-13T17:45:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d7fff640 1 -- 172.21.15.96:0/1953611483 learned_addr learned my addr 172.21.15.96:0/1953611483 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d77fe640 1 -- 172.21.15.96:0/1953611483 >> v1:172.21.15.96:6789/0 conn(0x7f87d809d690 legacy=0x7f87d8138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d77fe640 1 -- 172.21.15.96:0/1953611483 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d809cba0 con 0x7f87d813af50 2024-09-13T17:45:44.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d6ffd640 1 -- 172.21.15.96:0/1953611483 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431354352 0 0) 0x7f87c8001120 con 0x7f87d809d690 2024-09-13T17:45:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d77fe640 1 --2- 172.21.15.96:0/1953611483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d813af50 0x7f87d813d340 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009740 tx=0x7f87cc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1abdcacad9dc8fb server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d6ffd640 1 -- 172.21.15.96:0/1953611483 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87cc0a8840 con 0x7f87d813af50 2024-09-13T17:45:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d6ffd640 1 -- 172.21.15.96:0/1953611483 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0a89a0 con 0x7f87d813af50 2024-09-13T17:45:44.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.058+0000 7f87d6ffd640 1 -- 172.21.15.96:0/1953611483 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87cc0a8c60 con 0x7f87d813af50 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1953611483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d813af50 msgr2=0x7f87d813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 --2- 172.21.15.96:0/1953611483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d813af50 0x7f87d813d340 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009740 tx=0x7f87cc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1953611483 shutdown_connections 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 --2- 172.21.15.96:0/1953611483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d813af50 0x7f87d813d340 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1953611483 >> 172.21.15.96:0/1953611483 conn(0x7f87d806bc60 msgr2=0x7f87d806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1953611483 shutdown_connections 2024-09-13T17:45:44.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1953611483 wait complete. 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 Processor -- start 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- start start 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80a6280 con 0x7f87d809d690 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39726/0 (socket says 172.21.15.96:39726) 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d7fff640 1 -- 172.21.15.96:0/1646893237 learned_addr learned my addr 172.21.15.96:0/1646893237 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:44.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d7fff640 1 -- 172.21.15.96:0/1646893237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc009420 con 0x7f87d809d690 2024-09-13T17:45:44.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d7fff640 1 --2- 172.21.15.96:0/1646893237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c390 tx=0x7f87c800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87c8007410 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8007590 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d80a51d0 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f87c80047f0 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.062+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d80a7170 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.066+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f87c8004950 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.066+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d80a5ee0 con 0x7f87d809d690 2024-09-13T17:45:44.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.066+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f87c8046f30 con 0x7f87d809d690 2024-09-13T17:45:44.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.074+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f87c801bd70 con 0x7f87d809d690 2024-09-13T17:45:44.181 INFO:tasks.ceph.mgr.x.smithi096.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-13T17:45:44.181 INFO:tasks.ceph.mgr.x.smithi096.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-13T17:45:44.181 INFO:tasks.ceph.mgr.x.smithi096.stderr: from numpy import show_config as show_numpy_config 2024-09-13T17:45:44.188 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:44.186+0000 7fe8513d1180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T17:45:44.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.270+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87d809bc00 con 0x7f87d809d690 2024-09-13T17:45:44.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.270+0000 7f87d4ff9640 1 -- 172.21.15.96:0/1646893237 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+10994 (secure 0 0 0) 0x7f87d809bc00 con 0x7f87d809d690 2024-09-13T17:45:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:44.273 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":70,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:42.332411+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2416365020":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/3625984202":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6825/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6824/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1018463101":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/928338259":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1348634400":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 msgr2=0x7f87d80a6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 --2- 172.21.15.96:0/1646893237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f87c800c390 tx=0x7f87c800c850 comp rx=0 tx=0).stop 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 shutdown_connections 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 --2- 172.21.15.96:0/1646893237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87d809d690 0x7f87d80a6c30 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 >> 172.21.15.96:0/1646893237 conn(0x7f87d806bc60 msgr2=0x7f87d8139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 shutdown_connections 2024-09-13T17:45:44.275 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.274+0000 7f87de951640 1 -- 172.21.15.96:0/1646893237 wait complete. 2024-09-13T17:45:44.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-13T17:41:51.865891+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.6700000166893005, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-13T17:45:25.156688+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.409999966621399, 'score_stable': 1.409999966621399, 'optimal_score': 1, 'raw_score_acting': 1.409999966621399, 'raw_score_stable': 1.409999966621399, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-09-13T17:45:25.809998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-13T17:45:44.292 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-13T17:45:44.324 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:44.322+0000 7fe8513d1180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:44.436 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:44.434+0000 7fe8513d1180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T17:45:44.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.446+0000 7f982d5e5640 1 Processor -- start 2024-09-13T17:45:44.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.446+0000 7f982d5e5640 1 -- start start 2024-09-13T17:45:44.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9828135140 0x7f9828135520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f982812ef00 con 0x7f982809c960 2024-09-13T17:45:44.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9828099ec0 con 0x7f9828135140 2024-09-13T17:45:44.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f9826ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9828135140 0x7f9828135520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f981e7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f982809c960 0x7f9828131440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55604/0 (socket says 172.21.15.96:55604) 2024-09-13T17:45:44.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f981e7fc640 1 -- 172.21.15.96:0/3563755346 learned_addr learned my addr 172.21.15.96:0/3563755346 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:44.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f9826ffd640 1 -- 172.21.15.96:0/3563755346 >> v1:172.21.15.96:6789/0 conn(0x7f982809c960 legacy=0x7f9828131440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f98267fc640 1 -- 172.21.15.96:0/3563755346 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439508685 0 0) 0x7f9810001120 con 0x7f982809c960 2024-09-13T17:45:44.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f9826ffd640 1 -- 172.21.15.96:0/3563755346 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f982809bf20 con 0x7f9828135140 2024-09-13T17:45:44.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f9826ffd640 1 --2- 172.21.15.96:0/3563755346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9828135140 0x7f9828135520 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9818009740 tx=0x7f981801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd29d5bf96a0013 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f98267fc640 1 -- 172.21.15.96:0/3563755346 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9828132360 con 0x7f9828135140 2024-09-13T17:45:44.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f98267fc640 1 -- 172.21.15.96:0/3563755346 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98180a7050 con 0x7f9828135140 2024-09-13T17:45:44.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f98267fc640 1 -- 172.21.15.96:0/3563755346 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98180ab680 con 0x7f9828135140 2024-09-13T17:45:44.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- 172.21.15.96:0/3563755346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9828135140 msgr2=0x7f9828135520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 --2- 172.21.15.96:0/3563755346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9828135140 0x7f9828135520 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f9818009740 tx=0x7f981801c420 comp rx=0 tx=0).stop 2024-09-13T17:45:44.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- 172.21.15.96:0/3563755346 shutdown_connections 2024-09-13T17:45:44.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 --2- 172.21.15.96:0/3563755346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9828135140 0x7f9828135520 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- 172.21.15.96:0/3563755346 >> 172.21.15.96:0/3563755346 conn(0x7f982806e020 msgr2=0x7f9828070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.450+0000 7f982d5e5640 1 -- 172.21.15.96:0/3563755346 shutdown_connections 2024-09-13T17:45:44.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- 172.21.15.96:0/3563755346 wait complete. 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 Processor -- start 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- start start 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98280a45a0 con 0x7f982809c960 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f9826ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f9826ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39754/0 (socket says 172.21.15.96:39754) 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f9826ffd640 1 -- 172.21.15.96:0/2420092906 learned_addr learned my addr 172.21.15.96:0/2420092906 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f9826ffd640 1 -- 172.21.15.96:0/2420092906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818009420 con 0x7f982809c960 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f9826ffd640 1 --2- 172.21.15.96:0/2420092906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f981801a050 tx=0x7f981801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98180abea0 con 0x7f982809c960 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9818004020 con 0x7f982809c960 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9828133920 con 0x7f982809c960 2024-09-13T17:45:44.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98180042c0 con 0x7f982809c960 2024-09-13T17:45:44.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98280a42e0 con 0x7f982809c960 2024-09-13T17:45:44.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f98180b2050 con 0x7f982809c960 2024-09-13T17:45:44.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f98180e16e0 con 0x7f982809c960 2024-09-13T17:45:44.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.454+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98280a46e0 con 0x7f982809c960 2024-09-13T17:45:44.462 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.462+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f98280a46e0 con 0x7f982809c960 2024-09-13T17:45:44.500 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:44.498+0000 7fe8513d1180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T17:45:44.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.618+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f98280a46e0 con 0x7f982809c960 2024-09-13T17:45:44.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.618+0000 7f981ffff640 1 -- 172.21.15.96:0/2420092906 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f98280a46e0 con 0x7f982809c960 2024-09-13T17:45:44.620 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-09-13T17:45:44.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.618+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 msgr2=0x7f98280a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.618+0000 7f982d5e5640 1 --2- 172.21.15.96:0/2420092906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f981801a050 tx=0x7f981801ac60 comp rx=0 tx=0).stop 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.622+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 shutdown_connections 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.622+0000 7f982d5e5640 1 --2- 172.21.15.96:0/2420092906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f982809c960 0x7f98280a07e0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.622+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 >> 172.21.15.96:0/2420092906 conn(0x7f982806e020 msgr2=0x7f9828134430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.622+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 shutdown_connections 2024-09-13T17:45:44.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.622+0000 7f982d5e5640 1 -- 172.21.15.96:0/2420092906 wait complete. 2024-09-13T17:45:44.634 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:44.630+0000 7fe8513d1180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T17:45:44.638 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-13T17:45:44.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.750+0000 7f3eea6bc640 1 Processor -- start 2024-09-13T17:45:44.754 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.750+0000 7f3eea6bc640 1 -- start start 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3ee409b0a0 0x7f3ee41350a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee4135a30 con 0x7f3ee41355e0 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee4134af0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3ee409b0a0 0x7f3ee41350a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee37fe640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3ee41355e0 0x7f3ee4131540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55606/0 (socket says 172.21.15.96:55606) 2024-09-13T17:45:44.755 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee3fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3ee409b0a0 0x7f3ee41350a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39756/0 (socket says 172.21.15.96:39756) 2024-09-13T17:45:44.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee37fe640 1 -- 172.21.15.96:0/3741391761 learned_addr learned my addr 172.21.15.96:0/3741391761 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:44.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee3fff640 1 -- 172.21.15.96:0/3741391761 >> v1:172.21.15.96:6789/0 conn(0x7f3ee41355e0 legacy=0x7f3ee4131540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.756 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee3fff640 1 -- 172.21.15.96:0/3741391761 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee409b480 con 0x7f3ee409b0a0 2024-09-13T17:45:44.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee3fff640 1 --2- 172.21.15.96:0/3741391761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41350a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009740 tx=0x7f3ed401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596b28f3aef014da server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee2ffd640 1 -- 172.21.15.96:0/3741391761 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ee4132460 con 0x7f3ee409b0a0 2024-09-13T17:45:44.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee2ffd640 1 -- 172.21.15.96:0/3741391761 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed40a7050 con 0x7f3ee409b0a0 2024-09-13T17:45:44.757 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3ee2ffd640 1 -- 172.21.15.96:0/3741391761 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ed40042f0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- 172.21.15.96:0/3741391761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 msgr2=0x7f3ee41350a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 --2- 172.21.15.96:0/3741391761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41350a0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4009740 tx=0x7f3ed401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- 172.21.15.96:0/3741391761 shutdown_connections 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 --2- 172.21.15.96:0/3741391761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41350a0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- 172.21.15.96:0/3741391761 >> 172.21.15.96:0/3741391761 conn(0x7f3ee406e020 msgr2=0x7f3ee4070440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- 172.21.15.96:0/3741391761 shutdown_connections 2024-09-13T17:45:44.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.754+0000 7f3eea6bc640 1 -- 172.21.15.96:0/3741391761 wait complete. 2024-09-13T17:45:44.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 Processor -- start 2024-09-13T17:45:44.759 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 -- start start 2024-09-13T17:45:44.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:44.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee413a980 con 0x7f3ee409b0a0 2024-09-13T17:45:44.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:44.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39762/0 (socket says 172.21.15.96:39762) 2024-09-13T17:45:44.760 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee3fff640 1 -- 172.21.15.96:0/2111846540 learned_addr learned my addr 172.21.15.96:0/2111846540 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee3fff640 1 -- 172.21.15.96:0/2111846540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4009420 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee3fff640 1 --2- 172.21.15.96:0/2111846540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4018fa0 tx=0x7f3ed4019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ed40039e0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed4003b40 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee412ef20 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ed4003de0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee41cd600 con 0x7f3ee409b0a0 2024-09-13T17:45:44.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f3ed40b2040 con 0x7f3ee409b0a0 2024-09-13T17:45:44.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ee41350a0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.758+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f3ed40e1610 con 0x7f3ee409b0a0 2024-09-13T17:45:44.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.762+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3ed40aa7b0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.918+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ee409e2a0 con 0x7f3ee409b0a0 2024-09-13T17:45:44.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3ee0ff9640 1 -- 172.21.15.96:0/2111846540 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v70) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ed40b0070 con 0x7f3ee409b0a0 2024-09-13T17:45:44.923 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 msgr2=0x7f3ee41cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 --2- 172.21.15.96:0/2111846540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3ed4018fa0 tx=0x7f3ed4019130 comp rx=0 tx=0).stop 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 shutdown_connections 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 --2- 172.21.15.96:0/2111846540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ee409b0a0 0x7f3ee41cda30 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 >> 172.21.15.96:0/2111846540 conn(0x7f3ee406e020 msgr2=0x7f3ee406a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 shutdown_connections 2024-09-13T17:45:44.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:44.922+0000 7f3eea6bc640 1 -- 172.21.15.96:0/2111846540 wait complete. 2024-09-13T17:45:44.940 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-13T17:45:45.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.050+0000 7f9e04e5e640 1 Processor -- start 2024-09-13T17:45:45.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.050+0000 7f9e04e5e640 1 -- start start 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.050+0000 7f9e04e5e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9e0013af50 0x7f9e0013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.050+0000 7f9e04e5e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e00138050 con 0x7f9e0009d690 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.050+0000 7f9e04e5e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0009c410 con 0x7f9e0013af50 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfdd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f9e0013af50 0x7f9e0013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfe59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f9e0009d690 0x7f9e00138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55616/0 (socket says 172.21.15.96:55616) 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfe59c640 1 -- 172.21.15.96:0/4256159950 learned_addr learned my addr 172.21.15.96:0/4256159950 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfdd9b640 1 -- 172.21.15.96:0/4256159950 >> v1:172.21.15.96:6789/0 conn(0x7f9e0009d690 legacy=0x7f9e00138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfdd9b640 1 -- 172.21.15.96:0/4256159950 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0009cba0 con 0x7f9e0013af50 2024-09-13T17:45:45.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfd59a640 1 -- 172.21.15.96:0/4256159950 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128929779 0 0) 0x7f9df0001120 con 0x7f9e0009d690 2024-09-13T17:45:45.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfdd9b640 1 --2- 172.21.15.96:0/4256159950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0013af50 0x7f9e0013d340 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009740 tx=0x7f9df401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70190d7d3926db server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfd59a640 1 -- 172.21.15.96:0/4256159950 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df40a8840 con 0x7f9e0013af50 2024-09-13T17:45:45.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfd59a640 1 -- 172.21.15.96:0/4256159950 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df40a89a0 con 0x7f9e0013af50 2024-09-13T17:45:45.057 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9dfd59a640 1 -- 172.21.15.96:0/4256159950 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df40a8c60 con 0x7f9e0013af50 2024-09-13T17:45:45.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 -- 172.21.15.96:0/4256159950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0013af50 msgr2=0x7f9e0013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 --2- 172.21.15.96:0/4256159950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0013af50 0x7f9e0013d340 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f9df4009740 tx=0x7f9df401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:45.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 -- 172.21.15.96:0/4256159950 shutdown_connections 2024-09-13T17:45:45.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 --2- 172.21.15.96:0/4256159950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0013af50 0x7f9e0013d340 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 -- 172.21.15.96:0/4256159950 >> 172.21.15.96:0/4256159950 conn(0x7f9e0006bc60 msgr2=0x7f9e0006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 -- 172.21.15.96:0/4256159950 shutdown_connections 2024-09-13T17:45:45.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.054+0000 7f9e04e5e640 1 -- 172.21.15.96:0/4256159950 wait complete. 2024-09-13T17:45:45.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 Processor -- start 2024-09-13T17:45:45.060 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 -- start start 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000a5c90 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9dfe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9dfe59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39782/0 (socket says 172.21.15.96:39782) 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9dfe59c640 1 -- 172.21.15.96:0/2767041139 learned_addr learned my addr 172.21.15.96:0/2767041139 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9dfe59c640 1 -- 172.21.15.96:0/2767041139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4009420 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9dfe59c640 1 --2- 172.21.15.96:0/2767041139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9df000c390 tx=0x7f9df000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df0007410 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df0007590 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e000a6f20 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9df00047f0 con 0x7f9e0009d690 2024-09-13T17:45:45.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e00137400 con 0x7f9e0009d690 2024-09-13T17:45:45.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f9df0004950 con 0x7f9e0009d690 2024-09-13T17:45:45.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f9df0046f30 con 0x7f9e0009d690 2024-09-13T17:45:45.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.058+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e00138050 con 0x7f9e0009d690 2024-09-13T17:45:45.066 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.062+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f9e00138050 con 0x7f9e0009d690 2024-09-13T17:45:45.127 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.126+0000 7fe8513d1180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T17:45:45.200 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.198+0000 7fe8513d1180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T17:45:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e0009bc00 con 0x7f9e0009d690 2024-09-13T17:45:45.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9deeffd640 1 -- 172.21.15.96:0/2767041139 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v70) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e0009bc00 con 0x7f9e0009d690 2024-09-13T17:45:45.216 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 msgr2=0x7f9e000a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 --2- 172.21.15.96:0/2767041139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9df000c390 tx=0x7f9df000c850 comp rx=0 tx=0).stop 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 shutdown_connections 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 --2- 172.21.15.96:0/2767041139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e0009d690 0x7f9e000a69e0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 >> 172.21.15.96:0/2767041139 conn(0x7f9e0006bc60 msgr2=0x7f9e0009f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 shutdown_connections 2024-09-13T17:45:45.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.214+0000 7f9e04e5e640 1 -- 172.21.15.96:0/2767041139 wait complete. 2024-09-13T17:45:45.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:45.347 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.346+0000 7fe8513d1180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T17:45:45.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.354+0000 7f3c357f7640 1 Processor -- start 2024-09-13T17:45:45.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.354+0000 7f3c357f7640 1 -- start start 2024-09-13T17:45:45.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f3c300a6510 0x7f3c3013af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c30070600 con 0x7f3c300a4900 2024-09-13T17:45:45.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c30070830 con 0x7f3c300a6510 2024-09-13T17:45:45.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2effd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f3c300a4900 0x7f3c300a4ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55624/0 (socket says 172.21.15.96:55624) 2024-09-13T17:45:45.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2effd640 1 -- 172.21.15.96:0/2574599851 learned_addr learned my addr 172.21.15.96:0/2574599851 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:45.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2e7fc640 1 --2- 172.21.15.96:0/2574599851 >> v2:172.21.15.96:3300/0 conn(0x7f3c300a6510 0x7f3c3013af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2dffb640 1 -- 172.21.15.96:0/2574599851 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954918652 0 0) 0x7f3c30070600 con 0x7f3c300a4900 2024-09-13T17:45:45.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2dffb640 1 -- 172.21.15.96:0/2574599851 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c1c003530 con 0x7f3c300a4900 2024-09-13T17:45:45.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2e7fc640 1 -- 172.21.15.96:0/2574599851 >> v1:172.21.15.96:6789/0 conn(0x7f3c300a4900 legacy=0x7f3c300a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2e7fc640 1 -- 172.21.15.96:0/2574599851 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300a5c70 con 0x7f3c300a6510 2024-09-13T17:45:45.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2e7fc640 1 --2- 172.21.15.96:0/2574599851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a6510 0x7f3c3013af50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009740 tx=0x7f3c1801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8849863084edf2dc server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2dffb640 1 -- 172.21.15.96:0/2574599851 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c180ab530 con 0x7f3c300a6510 2024-09-13T17:45:45.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2dffb640 1 -- 172.21.15.96:0/2574599851 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c180a7050 con 0x7f3c300a6510 2024-09-13T17:45:45.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c2dffb640 1 -- 172.21.15.96:0/2574599851 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c180ab7c0 con 0x7f3c300a6510 2024-09-13T17:45:45.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- 172.21.15.96:0/2574599851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a6510 msgr2=0x7f3c3013af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 --2- 172.21.15.96:0/2574599851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a6510 0x7f3c3013af50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009740 tx=0x7f3c1801c420 comp rx=0 tx=0).stop 2024-09-13T17:45:45.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- 172.21.15.96:0/2574599851 shutdown_connections 2024-09-13T17:45:45.362 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 --2- 172.21.15.96:0/2574599851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a6510 0x7f3c3013af50 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- 172.21.15.96:0/2574599851 >> 172.21.15.96:0/2574599851 conn(0x7f3c3006a150 msgr2=0x7f3c3006a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- 172.21.15.96:0/2574599851 shutdown_connections 2024-09-13T17:45:45.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.358+0000 7f3c357f7640 1 -- 172.21.15.96:0/2574599851 wait complete. 2024-09-13T17:45:45.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 Processor -- start 2024-09-13T17:45:45.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 -- start start 2024-09-13T17:45:45.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c30098a60 con 0x7f3c300a4900 2024-09-13T17:45:45.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c2effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39804/0 (socket says 172.21.15.96:39804) 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c2effd640 1 -- 172.21.15.96:0/1393507055 learned_addr learned my addr 172.21.15.96:0/1393507055 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c2effd640 1 -- 172.21.15.96:0/1393507055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18009420 con 0x7f3c300a4900 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c2effd640 1 --2- 172.21.15.96:0/1393507055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c3d0 tx=0x7f3c2000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c20005ce0 con 0x7f3c300a4900 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20005e40 con 0x7f3c300a4900 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3c20007220 con 0x7f3c300a4900 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3009b7d0 con 0x7f3c300a4900 2024-09-13T17:45:45.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3009ba90 con 0x7f3c300a4900 2024-09-13T17:45:45.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c300a4d60 con 0x7f3c300a4900 2024-09-13T17:45:45.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f3c20007440 con 0x7f3c300a4900 2024-09-13T17:45:45.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.362+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f3c200469e0 con 0x7f3c300a4900 2024-09-13T17:45:45.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.366+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f3c20019b40 con 0x7f3c300a4900 2024-09-13T17:45:45.424 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.422+0000 7fe8513d1180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T17:45:45.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.546+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c300668c0 con 0x7f3c300a4900 2024-09-13T17:45:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.546+0000 7f3c277fe640 1 -- 172.21.15.96:0/1393507055 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f3c20017070 con 0x7f3c300a4900 2024-09-13T17:45:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:45.550 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:45.550 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 msgr2=0x7f3c3009bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 --2- 172.21.15.96:0/1393507055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c3d0 tx=0x7f3c2000c890 comp rx=0 tx=0).stop 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 shutdown_connections 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 --2- 172.21.15.96:0/1393507055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c300a4900 0x7f3c3009bf20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 >> 172.21.15.96:0/1393507055 conn(0x7f3c3006a150 msgr2=0x7f3c30132060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 shutdown_connections 2024-09-13T17:45:45.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.550+0000 7f3c357f7640 1 -- 172.21.15.96:0/1393507055 wait complete. 2024-09-13T17:45:45.567 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:45.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.678+0000 7f05190b9640 1 Processor -- start 2024-09-13T17:45:45.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f05190b9640 1 -- start start 2024-09-13T17:45:45.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f05190b9640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f051413af50 0x7f051413d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f05190b9640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0514138050 con 0x7f051409d690 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f05190b9640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f051409c410 con 0x7f051413af50 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f050a59c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f051413af50 0x7f051413d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f0512d9d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f051409d690 0x7f0514138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55626/0 (socket says 172.21.15.96:55626) 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f0512d9d640 1 -- 172.21.15.96:0/2151540188 learned_addr learned my addr 172.21.15.96:0/2151540188 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f050a59c640 1 -- 172.21.15.96:0/2151540188 >> v1:172.21.15.96:6789/0 conn(0x7f051409d690 legacy=0x7f0514138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.684 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f050a59c640 1 -- 172.21.15.96:0/2151540188 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051409cba0 con 0x7f051413af50 2024-09-13T17:45:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f050a59c640 1 --2- 172.21.15.96:0/2151540188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051413af50 0x7f051413d340 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009740 tx=0x7f04fc01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d28caeaada1fd07 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f051259c640 1 -- 172.21.15.96:0/2151540188 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04fc0a8840 con 0x7f051413af50 2024-09-13T17:45:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f051259c640 1 -- 172.21.15.96:0/2151540188 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04fc0a89a0 con 0x7f051413af50 2024-09-13T17:45:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.682+0000 7f051259c640 1 -- 172.21.15.96:0/2151540188 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f04fc0a8c60 con 0x7f051413af50 2024-09-13T17:45:45.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/2151540188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051413af50 msgr2=0x7f051413d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 --2- 172.21.15.96:0/2151540188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051413af50 0x7f051413d340 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f04fc009740 tx=0x7f04fc01ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/2151540188 shutdown_connections 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 --2- 172.21.15.96:0/2151540188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051413af50 0x7f051413d340 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/2151540188 >> 172.21.15.96:0/2151540188 conn(0x7f051406bc60 msgr2=0x7f051406c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/2151540188 shutdown_connections 2024-09-13T17:45:45.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/2151540188 wait complete. 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 Processor -- start 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- start start 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140a2d40 con 0x7f051409d690 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f0512d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f0512d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39818/0 (socket says 172.21.15.96:39818) 2024-09-13T17:45:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f0512d9d640 1 -- 172.21.15.96:0/3131228123 learned_addr learned my addr 172.21.15.96:0/3131228123 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f0512d9d640 1 -- 172.21.15.96:0/3131228123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc009420 con 0x7f051409d690 2024-09-13T17:45:45.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f0512d9d640 1 --2- 172.21.15.96:0/3131228123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f050400c380 tx=0x7f050400c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0504005c20 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0504005d80 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05141372e0 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0504007200 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140a5100 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05140a4f10 con 0x7f051409d690 2024-09-13T17:45:45.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.686+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 35) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f0504007360 con 0x7f051409d690 2024-09-13T17:45:45.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.690+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f0504046f60 con 0x7f051409d690 2024-09-13T17:45:45.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.694+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0504072db0 con 0x7f051409d690 2024-09-13T17:45:45.715 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.714+0000 7fe8513d1180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T17:45:45.770 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.766+0000 7fe8513d1180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T17:45:45.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.794+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 36) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f05040454e0 con 0x7f051409d690 2024-09-13T17:45:45.863 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7d9944640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.863 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7d9944640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.863 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7d9944640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.863 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7d9944640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.863 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7d9944640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.862+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0514138050 con 0x7f051409d690 2024-09-13T17:45:45.864 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7dc94a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.864 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7dc94a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.864 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7dc94a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.864 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7dc94a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.865 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:45.862+0000 7fe7dc94a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-13T17:45:45.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.862+0000 7f050bfff640 1 -- 172.21.15.96:0/3131228123 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+10994 (secure 0 0 0) 0x7f0514138050 con 0x7f051409d690 2024-09-13T17:45:45.865 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:45.865 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":71,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:45.788365+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2416365020":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/3625984202":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6825/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6824/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1018463101":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/928338259":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1348634400":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 msgr2=0x7f05141c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 --2- 172.21.15.96:0/3131228123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f050400c380 tx=0x7f050400c840 comp rx=0 tx=0).stop 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 shutdown_connections 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 --2- 172.21.15.96:0/3131228123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f051409d690 0x7f05141c9550 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 >> 172.21.15.96:0/3131228123 conn(0x7f051406bc60 msgr2=0x7f0514131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 shutdown_connections 2024-09-13T17:45:45.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.866+0000 7f05190b9640 1 -- 172.21.15.96:0/3131228123 wait complete. 2024-09-13T17:45:45.886 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:45:45.887 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:45.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.986+0000 7fb8beed6640 1 Processor -- start 2024-09-13T17:45:45.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 -- start start 2024-09-13T17:45:45.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8b81375b0 0x7fb8b81399a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b8135100 con 0x7fb8b8133600 2024-09-13T17:45:45.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b806dad0 con 0x7fb8b81375b0 2024-09-13T17:45:45.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b7fff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fb8b81375b0 0x7fb8b81399a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8bcc72640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fb8b8133600 0x7fb8b81339e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55640/0 (socket says 172.21.15.96:55640) 2024-09-13T17:45:45.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8bcc72640 1 -- 172.21.15.96:0/782085533 learned_addr learned my addr 172.21.15.96:0/782085533 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:45.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b7fff640 1 -- 172.21.15.96:0/782085533 >> v1:172.21.15.96:6789/0 conn(0x7fb8b8133600 legacy=0x7fb8b81339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b7fff640 1 -- 172.21.15.96:0/782085533 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b806eb80 con 0x7fb8b81375b0 2024-09-13T17:45:45.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b7fff640 1 --2- 172.21.15.96:0/782085533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b81375b0 0x7fb8b81399a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009740 tx=0x7fb8a401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2e9a3828669f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b77fe640 1 -- 172.21.15.96:0/782085533 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a40ab530 con 0x7fb8b81375b0 2024-09-13T17:45:45.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b77fe640 1 -- 172.21.15.96:0/782085533 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40a7050 con 0x7fb8b81375b0 2024-09-13T17:45:45.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8b77fe640 1 -- 172.21.15.96:0/782085533 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8a40ab7e0 con 0x7fb8b81375b0 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 -- 172.21.15.96:0/782085533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b81375b0 msgr2=0x7fb8b81399a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.990+0000 7fb8beed6640 1 --2- 172.21.15.96:0/782085533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b81375b0 0x7fb8b81399a0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009740 tx=0x7fb8a401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/782085533 shutdown_connections 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 --2- 172.21.15.96:0/782085533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b81375b0 0x7fb8b81399a0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/782085533 >> 172.21.15.96:0/782085533 conn(0x7fb8b806a150 msgr2=0x7fb8b806a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/782085533 shutdown_connections 2024-09-13T17:45:45.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/782085533 wait complete. 2024-09-13T17:45:45.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 Processor -- start 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- start start 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b811da00 con 0x7fb8b8133600 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8bcc72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8bcc72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39916/0 (socket says 172.21.15.96:39916) 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8bcc72640 1 -- 172.21.15.96:0/2095931789 learned_addr learned my addr 172.21.15.96:0/2095931789 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8bcc72640 1 -- 172.21.15.96:0/2095931789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4009420 con 0x7fb8b8133600 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8bcc72640 1 --2- 172.21.15.96:0/2095931789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c480 tx=0x7fb8ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8ac005c20 con 0x7fb8b8133600 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac005d80 con 0x7fb8b8133600 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b811ebb0 con 0x7fb8b8133600 2024-09-13T17:45:45.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8ac0071a0 con 0x7fb8b8133600 2024-09-13T17:45:45.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b8136ae0 con 0x7fb8b8133600 2024-09-13T17:45:45.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8073570 con 0x7fb8b8133600 2024-09-13T17:45:45.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 43018+0+0 (secure 0 0 0) 0x7fb8ac007300 con 0x7fb8b8133600 2024-09-13T17:45:45.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.994+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 3783+0+0 (secure 0 0 0) 0x7fb8ac046840 con 0x7fb8b8133600 2024-09-13T17:45:46.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:45.998+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fb8ac0075b0 con 0x7fb8b8133600 2024-09-13T17:45:46.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.178+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8b811d050 con 0x7fb8b8133600 2024-09-13T17:45:46.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.178+0000 7fb8b57fa640 1 -- 172.21.15.96:0/2095931789 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7fb8ac017030 con 0x7fb8b8133600 2024-09-13T17:45:46.182 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:46.182 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:46.182 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 msgr2=0x7fb8b811e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 --2- 172.21.15.96:0/2095931789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac00c480 tx=0x7fb8ac00c940 comp rx=0 tx=0).stop 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 shutdown_connections 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 --2- 172.21.15.96:0/2095931789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8b8133600 0x7fb8b811e3f0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 >> 172.21.15.96:0/2095931789 conn(0x7fb8b806a150 msgr2=0x7fb8b8132560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 shutdown_connections 2024-09-13T17:45:46.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.182+0000 7fb8beed6640 1 -- 172.21.15.96:0/2095931789 wait complete. 2024-09-13T17:45:46.199 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:45:46.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.298+0000 7f3712e49640 1 Processor -- start 2024-09-13T17:45:46.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.298+0000 7f3712e49640 1 -- start start 2024-09-13T17:45:46.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f370c06f6d0 0x7f370c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f370c06e130 con 0x7f370c09dc20 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f370c0a6750 con 0x7f370c06f6d0 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3710be5640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f370c06f6d0 0x7f370c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f370bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f370c09dc20 0x7f370c134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55656/0 (socket says 172.21.15.96:55656) 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f370bfff640 1 -- 172.21.15.96:0/2610480881 learned_addr learned my addr 172.21.15.96:0/2610480881 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3710be5640 1 -- 172.21.15.96:0/2610480881 >> v1:172.21.15.96:6789/0 conn(0x7f370c09dc20 legacy=0x7f370c134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3710be5640 1 -- 172.21.15.96:0/2610480881 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f370c0712e0 con 0x7f370c06f6d0 2024-09-13T17:45:46.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3710be5640 1 --2- 172.21.15.96:0/2610480881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c134ab0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009740 tx=0x7f36fc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5a7571d0477d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f370b7fe640 1 -- 172.21.15.96:0/2610480881 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36fc0ab400 con 0x7f370c06f6d0 2024-09-13T17:45:46.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f370b7fe640 1 -- 172.21.15.96:0/2610480881 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0a7050 con 0x7f370c06f6d0 2024-09-13T17:45:46.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f370b7fe640 1 -- 172.21.15.96:0/2610480881 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36fc0ab6b0 con 0x7f370c06f6d0 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- 172.21.15.96:0/2610480881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 msgr2=0x7f370c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 --2- 172.21.15.96:0/2610480881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c134ab0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009740 tx=0x7f36fc01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- 172.21.15.96:0/2610480881 shutdown_connections 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 --2- 172.21.15.96:0/2610480881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c134ab0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- 172.21.15.96:0/2610480881 >> 172.21.15.96:0/2610480881 conn(0x7f370c098a60 msgr2=0x7f370c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- 172.21.15.96:0/2610480881 shutdown_connections 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.302+0000 7f3712e49640 1 -- 172.21.15.96:0/2610480881 wait complete. 2024-09-13T17:45:46.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 Processor -- start 2024-09-13T17:45:46.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 -- start start 2024-09-13T17:45:46.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370c138fd0 con 0x7f370c06f6d0 2024-09-13T17:45:46.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3710be5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3710be5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39934/0 (socket says 172.21.15.96:39934) 2024-09-13T17:45:46.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3710be5640 1 -- 172.21.15.96:0/3995008187 learned_addr learned my addr 172.21.15.96:0/3995008187 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:46.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3710be5640 1 -- 172.21.15.96:0/3995008187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009420 con 0x7f370c06f6d0 2024-09-13T17:45:46.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3710be5640 1 --2- 172.21.15.96:0/3995008187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f36fc019fc0 tx=0x7f36fc01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36fc0b7450 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36fc0b75b0 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370c11e2e0 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36fc0b7850 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f370c1384d0 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f370c134ab0 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f36fc0b79b0 con 0x7f370c06f6d0 2024-09-13T17:45:46.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.306+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f36fc0e1e60 con 0x7f370c06f6d0 2024-09-13T17:45:46.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.310+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f36fc0c76c0 con 0x7f370c06f6d0 2024-09-13T17:45:46.501 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.498+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f370c11dc80 con 0x7f370c06f6d0 2024-09-13T17:45:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.498+0000 7f37097fa640 1 -- 172.21.15.96:0/3995008187 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+3222 (secure 0 0 0) 0x7f36fc0b0070 con 0x7f370c06f6d0 2024-09-13T17:45:46.502 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:46.502 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":43,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-13T17:45:26.833191+0000","modified":"2024-09-13T17:45:29.843047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5371},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5371":{"gid":5371,"name":"b","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.96:6829/1571674225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":1571674225},{"type":"v1","addr":"172.21.15.96:6829","nonce":1571674225}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-09-13T17:45:46.502 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 43 2024-09-13T17:45:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 msgr2=0x7f370c136d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 --2- 172.21.15.96:0/3995008187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f36fc019fc0 tx=0x7f36fc01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 shutdown_connections 2024-09-13T17:45:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 --2- 172.21.15.96:0/3995008187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f370c06f6d0 0x7f370c136d90 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 >> 172.21.15.96:0/3995008187 conn(0x7f370c098a60 msgr2=0x7f370c133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 shutdown_connections 2024-09-13T17:45:46.505 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.502+0000 7f3712e49640 1 -- 172.21.15.96:0/3995008187 wait complete. 2024-09-13T17:45:46.520 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:46.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f83195fa640 1 Processor -- start 2024-09-13T17:45:46.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f83195fa640 1 -- start start 2024-09-13T17:45:46.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f83195fa640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8314134670 0x7f8314134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f83195fa640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8314071670 con 0x7f831406e5c0 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f83195fa640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8314071360 con 0x7f8314134670 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f8312ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8314134670 0x7f8314134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f830a7fc640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f831406e5c0 0x7f831406ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55670/0 (socket says 172.21.15.96:55670) 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.630+0000 7f830a7fc640 1 -- 172.21.15.96:0/2101641781 learned_addr learned my addr 172.21.15.96:0/2101641781 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f8312ffd640 1 -- 172.21.15.96:0/2101641781 >> v1:172.21.15.96:6789/0 conn(0x7f831406e5c0 legacy=0x7f831406ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f8312ffd640 1 -- 172.21.15.96:0/2101641781 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83140a7200 con 0x7f8314134670 2024-09-13T17:45:46.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83127fc640 1 -- 172.21.15.96:0/2101641781 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076354000 0 0) 0x7f8314071670 con 0x7f831406e5c0 2024-09-13T17:45:46.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f8312ffd640 1 --2- 172.21.15.96:0/2101641781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8314134670 0x7f8314134a50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8304009740 tx=0x7f830401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75a7ddd8e9427ea server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83127fc640 1 -- 172.21.15.96:0/2101641781 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83040ab530 con 0x7f8314134670 2024-09-13T17:45:46.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83127fc640 1 -- 172.21.15.96:0/2101641781 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040a7050 con 0x7f8314134670 2024-09-13T17:45:46.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83127fc640 1 -- 172.21.15.96:0/2101641781 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83040ab7e0 con 0x7f8314134670 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 -- 172.21.15.96:0/2101641781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8314134670 msgr2=0x7f8314134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 --2- 172.21.15.96:0/2101641781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8314134670 0x7f8314134a50 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8304009740 tx=0x7f830401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 -- 172.21.15.96:0/2101641781 shutdown_connections 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 --2- 172.21.15.96:0/2101641781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8314134670 0x7f8314134a50 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 -- 172.21.15.96:0/2101641781 >> 172.21.15.96:0/2101641781 conn(0x7f8314098a60 msgr2=0x7f831409ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 -- 172.21.15.96:0/2101641781 shutdown_connections 2024-09-13T17:45:46.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.634+0000 7f83195fa640 1 -- 172.21.15.96:0/2101641781 wait complete. 2024-09-13T17:45:46.639 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 Processor -- start 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 -- start start 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83140a1fb0 con 0x7f831406e5c0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f8312ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f8312ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39950/0 (socket says 172.21.15.96:39950) 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f8312ffd640 1 -- 172.21.15.96:0/1678926794 learned_addr learned my addr 172.21.15.96:0/1678926794 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f8312ffd640 1 -- 172.21.15.96:0/1678926794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304009420 con 0x7f831406e5c0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f8312ffd640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8304018fa0 tx=0x7f830401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83040b7660 con 0x7f831406e5c0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83040b77e0 con 0x7f831406e5c0 2024-09-13T17:45:46.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83140a3240 con 0x7f831406e5c0 2024-09-13T17:45:46.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f83040b7ac0 con 0x7f831406e5c0 2024-09-13T17:45:46.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83141394c0 con 0x7f831406e5c0 2024-09-13T17:45:46.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 36) v1 ==== 43018+0+0 (secure 0 0 0) 0x7f83040b7c20 con 0x7f831406e5c0 2024-09-13T17:45:46.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83140a0760 con 0x7f831406e5c0 2024-09-13T17:45:46.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.638+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f83040e1f50 con 0x7f831406e5c0 2024-09-13T17:45:46.645 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.642+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f83040c7950 con 0x7f831406e5c0 2024-09-13T17:45:46.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.786+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f831409d520 con 0x7f831406e5c0 2024-09-13T17:45:46.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.790+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+10994 (secure 0 0 0) 0x7f83040b0070 con 0x7f831406e5c0 2024-09-13T17:45:46.793 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:46.793 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":71,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:45.788365+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-13T17:45:25.156688+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.4099999666213989,"score_stable":1.4099999666213989,"optimal_score":1,"raw_score_acting":1.4099999666213989,"raw_score_stable":1.4099999666213989,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-13T17:45:25.809998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2416365020":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/3625984202":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6825/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6824/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1018463101":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/928338259":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1348634400":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:46.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 msgr2=0x7f83140a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 37) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f83040b58d0 con 0x7f831406e5c0 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8304018fa0 tx=0x7f830401abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f830bfff640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f82f4039920 0x7f82f403bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 shutdown_connections 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f82f4039920 0x7f82f403bd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f831406e5c0 0x7f83140a2d00 secure :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8304018fa0 tx=0x7f830401abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 >> 172.21.15.96:0/1678926794 conn(0x7f8314098a60 msgr2=0x7f831412f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f830bfff640 1 -- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f82f4039920 msgr2=0x7f82f403bd10 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-13T17:45:46.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f830bfff640 1 --2- 172.21.15.96:0/1678926794 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f82f4039920 0x7f82f403bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 shutdown_connections 2024-09-13T17:45:46.798 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.794+0000 7f83195fa640 1 -- 172.21.15.96:0/1678926794 wait complete. 2024-09-13T17:45:46.814 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-13T17:45:46.922 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 Processor -- start 2024-09-13T17:45:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- start start 2024-09-13T17:45:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ff8138850 0x7f0ff80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.923 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff80a00b0 con 0x7f0ff8137d60 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80a12d0 con 0x7f0ff8138850 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ff8138850 0x7f0ff80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f0ff8138850 0x7f0ff80a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39958/0 (socket says 172.21.15.96:39958) 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 -- 172.21.15.96:0/1725795391 learned_addr learned my addr 172.21.15.96:0/1725795391 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 -- 172.21.15.96:0/1725795391 >> v1:172.21.15.96:6789/0 conn(0x7f0ff8137d60 legacy=0x7f0ff8138140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.924 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 -- 172.21.15.96:0/1725795391 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80a0350 con 0x7f0ff8138850 2024-09-13T17:45:46.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff6ffd640 1 --2- 172.21.15.96:0/1725795391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8138850 0x7f0ff80a0d90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0097f0 tx=0x7f0fec01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf49b07a00c01c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.925 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff67fc640 1 -- 172.21.15.96:0/1725795391 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fec0ab530 con 0x7f0ff8138850 2024-09-13T17:45:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff67fc640 1 -- 172.21.15.96:0/1725795391 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec0a7050 con 0x7f0ff8138850 2024-09-13T17:45:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ff67fc640 1 -- 172.21.15.96:0/1725795391 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fec0ab7e0 con 0x7f0ff8138850 2024-09-13T17:45:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/1725795391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8138850 msgr2=0x7f0ff80a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/1725795391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8138850 0x7f0ff80a0d90 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0097f0 tx=0x7f0fec01c450 comp rx=0 tx=0).stop 2024-09-13T17:45:46.926 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/1725795391 shutdown_connections 2024-09-13T17:45:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/1725795391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8138850 0x7f0ff80a0d90 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/1725795391 >> 172.21.15.96:0/1725795391 conn(0x7f0ff806ef80 msgr2=0x7f0ff80713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/1725795391 shutdown_connections 2024-09-13T17:45:46.927 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.922+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/1725795391 wait complete. 2024-09-13T17:45:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 Processor -- start 2024-09-13T17:45:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 -- start start 2024-09-13T17:45:46.928 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80a72f0 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39962/0 (socket says 172.21.15.96:39962) 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff77fe640 1 -- 172.21.15.96:0/2035916947 learned_addr learned my addr 172.21.15.96:0/2035916947 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff77fe640 1 -- 172.21.15.96:0/2035916947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0094d0 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff77fe640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c480 tx=0x7f0fe800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fe8005150 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe80052b0 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff80a82a0 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0fe8007030 con 0x7f0ff8137d60 2024-09-13T17:45:46.929 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff80a6390 con 0x7f0ff8137d60 2024-09-13T17:45:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 37) v1 ==== 43145+0+0 (secure 0 0 0) 0x7f0fe80072a0 con 0x7f0ff8137d60 2024-09-13T17:45:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff80a6730 con 0x7f0ff8137d60 2024-09-13T17:45:46.930 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 0x7f0fe4037710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 3783+0+0 (secure 0 0 0) 0x7f0fe8046a50 con 0x7f0ff8137d60 2024-09-13T17:45:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff6ffd640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 0x7f0fe4037710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:46.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.926+0000 7f0ff6ffd640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 0x7f0fe4037710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ad000 tx=0x7f0fec0a6000 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:46.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:46.930+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0fe80108d0 con 0x7f0ff8137d60 2024-09-13T17:45:47.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.110+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0ff813c4f0 con 0x7f0ff8137d60 2024-09-13T17:45:47.114 INFO:tasks.ceph.mon.a.smithi096.stderr:2024-09-13T17:45:47.114+0000 7ffa4752a640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-13T17:45:47.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.122+0000 7f0fdffff640 1 -- 172.21.15.96:0/2035916947 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v44) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0fe80102e0 con 0x7f0ff8137d60 2024-09-13T17:45:47.126 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-13T17:45:47.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 msgr2=0x7f0fe4037710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:47.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 0x7f0fe4037710 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0ad000 tx=0x7f0fec0a6000 comp rx=0 tx=0).stop 2024-09-13T17:45:47.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 msgr2=0x7f0ff80a2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c480 tx=0x7f0fe800c940 comp rx=0 tx=0).stop 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 shutdown_connections 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f0fe4035250 0x7f0fe4037710 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 --2- 172.21.15.96:0/2035916947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ff8137d60 0x7f0ff80a2990 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 >> 172.21.15.96:0/2035916947 conn(0x7f0ff806ef80 msgr2=0x7f0ff81373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 shutdown_connections 2024-09-13T17:45:47.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.126+0000 7f0ffdcc4640 1 -- 172.21.15.96:0/2035916947 wait complete. 2024-09-13T17:45:47.129 INFO:tasks.ceph.mds.b.smithi096.stderr: -1505> 2024-09-13T17:45:28.346+0000 7fafbfbe1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-13T17:45:47.144 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-13T17:45:47.178 INFO:tasks.ceph.mds.b.smithi096.stdout:starting mds.b at 2024-09-13T17:45:47.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0eea6b640 1 Processor -- start 2024-09-13T17:45:47.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0eea6b640 1 -- start start 2024-09-13T17:45:47.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0eea6b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 0x7fd0e8132d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:47.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0eea6b640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e81327b0 con 0x7fd0e809dc30 2024-09-13T17:45:47.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0eea6b640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e809bd90 con 0x7fd0e81329b0 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e7fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fd0e809dc30 0x7fd0e8135980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55688/0 (socket says 172.21.15.96:55688) 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e7fff640 1 -- 172.21.15.96:0/245015210 learned_addr learned my addr 172.21.15.96:0/245015210 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0ec807640 1 --2- 172.21.15.96:0/245015210 >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 0x7fd0e8132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049369907 0 0) 0x7fd0e81327b0 con 0x7fd0e809dc30 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d0003530 con 0x7fd0e809dc30 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v1:172.21.15.96:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4234499407 0 0) 0x7fd0d0003530 con 0x7fd0e809dc30 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 msgr2=0x7fd0e8132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 --2- 172.21.15.96:0/245015210 >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 0x7fd0e8132d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 --> v1:172.21.15.96:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e809b2a0 con 0x7fd0e809dc30 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v1:172.21.15.96:6789/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (unknown 597839753 0 0) 0x7fd0dc0015b0 con 0x7fd0e809dc30 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0ec807640 1 --2- 172.21.15.96:0/245015210 >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 0x7fd0e8132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 >> v1:172.21.15.96:6789/0 conn(0x7fd0e809dc30 legacy=0x7fd0e8135980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 --2- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0d0003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:47.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00043a0 con 0x7fd0d0003980 2024-09-13T17:45:47.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0ec807640 1 --2- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0d0003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:47.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.254+0000 7fd0ec807640 1 -- 172.21.15.96:0/245015210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d0003530 con 0x7fd0d0003980 2024-09-13T17:45:47.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0ec807640 1 --2- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0d0003d80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e350 tx=0x7fd0d801cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bf27919c843ccf server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:47.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0d80a5c90 con 0x7fd0d0003980 2024-09-13T17:45:47.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d800f040 con 0x7fd0d0003980 2024-09-13T17:45:47.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0e77fe640 1 -- 172.21.15.96:0/245015210 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0d80ad440 con 0x7fd0d0003980 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 msgr2=0x7fd0d0003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0d0003d80 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e350 tx=0x7fd0d801cfc0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- 172.21.15.96:0/245015210 shutdown_connections 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/245015210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0d0003d80 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/245015210 >> v2:172.21.15.96:3300/0 conn(0x7fd0e81329b0 0x7fd0e8132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- 172.21.15.96:0/245015210 >> 172.21.15.96:0/245015210 conn(0x7fd0e806ef80 msgr2=0x7fd0e80713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- 172.21.15.96:0/245015210 shutdown_connections 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- 172.21.15.96:0/245015210 wait complete. 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 Processor -- start 2024-09-13T17:45:47.262 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.258+0000 7fd0eea6b640 1 -- start start 2024-09-13T17:45:47.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0eea6b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:47.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0eea6b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80a0c10 con 0x7fd0d0003980 2024-09-13T17:45:47.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0ec807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0ec807640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39994/0 (socket says 172.21.15.96:39994) 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0ec807640 1 -- 172.21.15.96:0/2541596474 learned_addr learned my addr 172.21.15.96:0/2541596474 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0ec807640 1 -- 172.21.15.96:0/2541596474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8009a70 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0ec807640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd0d801d000 tx=0x7fd0d0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0d80ad7a0 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d80ad900 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80a0910 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0d80adbc0 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e80a91f0 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 37) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fd0d80add20 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80a0eb0 con 0x7fd0d0003980 2024-09-13T17:45:47.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 0x7fd0d4037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:47.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fd0d80e2650 con 0x7fd0d0003980 2024-09-13T17:45:47.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e7fff640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 0x7fd0d4037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:47.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.262+0000 7fd0e7fff640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 0x7fd0d4037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000f70 tx=0x7fd0dc001440 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:47.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.266+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fd0d8009de0 con 0x7fd0d0003980 2024-09-13T17:45:47.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:47.454+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd0e809d340 con 0x7fd0d0003980 2024-09-13T17:45:47.853 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:47.850+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:48.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.122+0000 7fd0e57fa640 1 -- 172.21.15.96:0/2541596474 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0d80bd020 con 0x7fd0d0003980 2024-09-13T17:45:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 msgr2=0x7fd0d4037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 0x7fd0d4037760 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc000f70 tx=0x7fd0dc001440 comp rx=0 tx=0).stop 2024-09-13T17:45:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 msgr2=0x7fd0e812dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fd0d801d000 tx=0x7fd0d0005800 comp rx=0 tx=0).stop 2024-09-13T17:45:48.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 shutdown_connections 2024-09-13T17:45:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fd0d40352a0 0x7fd0d4037760 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 --2- 172.21.15.96:0/2541596474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd0d0003980 0x7fd0e812dfa0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 >> 172.21.15.96:0/2541596474 conn(0x7fd0e806ef80 msgr2=0x7fd0e812ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 shutdown_connections 2024-09-13T17:45:48.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.126+0000 7fd0eea6b640 1 -- 172.21.15.96:0/2541596474 wait complete. 2024-09-13T17:45:48.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-13T17:45:48.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.326+0000 7fbe200f3640 1 Processor -- start 2024-09-13T17:45:48.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.326+0000 7fbe200f3640 1 -- start start 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe200f3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe18136170 0x7fbe18136550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe200f3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe18135cd0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe200f3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe18099c90 con 0x7fbe18136170 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1de8f640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbe18136170 0x7fbe18136550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1d68e640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbe1809a0a0 0x7fbe18132650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55694/0 (socket says 172.21.15.96:55694) 2024-09-13T17:45:48.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1d68e640 1 -- 172.21.15.96:0/2588159263 learned_addr learned my addr 172.21.15.96:0/2588159263 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1de8f640 1 -- 172.21.15.96:0/2588159263 >> v1:172.21.15.96:6789/0 conn(0x7fbe1809a0a0 legacy=0x7fbe18132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1de8f640 1 -- 172.21.15.96:0/2588159263 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1809d980 con 0x7fbe18136170 2024-09-13T17:45:48.332 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1ce8d640 1 -- 172.21.15.96:0/2588159263 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975622669 0 0) 0x7fbe18135cd0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1de8f640 1 --2- 172.21.15.96:0/2588159263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe18136170 0x7fbe18136550 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009740 tx=0x7fbe1401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd961c093e03cd server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:48.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1ce8d640 1 -- 172.21.15.96:0/2588159263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe18133570 con 0x7fbe18136170 2024-09-13T17:45:48.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1ce8d640 1 -- 172.21.15.96:0/2588159263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe140a7050 con 0x7fbe18136170 2024-09-13T17:45:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.330+0000 7fbe1ce8d640 1 -- 172.21.15.96:0/2588159263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe140ab550 con 0x7fbe18136170 2024-09-13T17:45:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2588159263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe18136170 msgr2=0x7fbe18136550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2588159263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe18136170 0x7fbe18136550 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009740 tx=0x7fbe1401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:48.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2588159263 shutdown_connections 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2588159263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe18136170 0x7fbe18136550 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2588159263 >> 172.21.15.96:0/2588159263 conn(0x7fbe1809c210 msgr2=0x7fbe18071190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2588159263 shutdown_connections 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2588159263 wait complete. 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 Processor -- start 2024-09-13T17:45:48.335 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- start start 2024-09-13T17:45:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180a04a0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1de8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:48.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1de8f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40024/0 (socket says 172.21.15.96:40024) 2024-09-13T17:45:48.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1de8f640 1 -- 172.21.15.96:0/2695352948 learned_addr learned my addr 172.21.15.96:0/2695352948 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:48.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1de8f640 1 -- 172.21.15.96:0/2695352948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe14009420 con 0x7fbe1809a0a0 2024-09-13T17:45:48.337 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1de8f640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbe14018fa0 tx=0x7fbe1401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe140abd70 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe14004020 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18071110 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbe14004190 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1806c5d0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 37) v1 ==== 43145+0+0 (secure 0 0 0) 0x7fbe140b2050 con 0x7fbe1809a0a0 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 0x7fbdfc037760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:48.338 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 3869+0+0 (secure 0 0 0) 0x7fbe140e1860 con 0x7fbe1809a0a0 2024-09-13T17:45:48.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe180a05e0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.334+0000 7fbe1d68e640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 0x7fbdfc037760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:48.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.338+0000 7fbe1d68e640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 0x7fbdfc037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007fd0 tx=0x7fbe08007920 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:48.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.342+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fbe180a05e0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.510+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe180a05e0 con 0x7fbe1809a0a0 2024-09-13T17:45:48.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:48.798+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fbe14003b40 con 0x7fbe1809a0a0 2024-09-13T17:45:48.821 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:48.818+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:49.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.126+0000 7fbe0e7fc640 1 -- 172.21.15.96:0/2695352948 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbe140b2de0 con 0x7fbe1809a0a0 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 msgr2=0x7fbdfc037760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 0x7fbdfc037760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fbe08007fd0 tx=0x7fbe08007920 comp rx=0 tx=0).stop 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 msgr2=0x7fbe1813b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fbe14018fa0 tx=0x7fbe1401abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 shutdown_connections 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbdfc0352a0 0x7fbdfc037760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 --2- 172.21.15.96:0/2695352948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe1809a0a0 0x7fbe1813b610 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:49.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 >> 172.21.15.96:0/2695352948 conn(0x7fbe1809c210 msgr2=0x7fbe1812c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:49.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 shutdown_connections 2024-09-13T17:45:49.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.134+0000 7fbe200f3640 1 -- 172.21.15.96:0/2695352948 wait complete. 2024-09-13T17:45:49.201 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-13T17:45:49.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d7176640 1 Processor -- start 2024-09-13T17:45:49.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d7176640 1 -- start start 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d7176640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f45d0134e00 0x7f45d01351e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d7176640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d0071cd0 con 0x7f45d006f6d0 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d7176640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00a6730 con 0x7f45d0134e00 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d4f12640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f45d0134e00 0x7f45d01351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45cffff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f45d006f6d0 0x7f45d006fab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55702/0 (socket says 172.21.15.96:55702) 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45cffff640 1 -- 172.21.15.96:0/962794150 learned_addr learned my addr 172.21.15.96:0/962794150 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d4f12640 1 -- 172.21.15.96:0/962794150 >> v1:172.21.15.96:6789/0 conn(0x7f45d006f6d0 legacy=0x7f45d006fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:49.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45cf7fe640 1 -- 172.21.15.96:0/962794150 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3830454140 0 0) 0x7f45d0071cd0 con 0x7f45d006f6d0 2024-09-13T17:45:49.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.338+0000 7f45d4f12640 1 -- 172.21.15.96:0/962794150 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0071140 con 0x7f45d0134e00 2024-09-13T17:45:49.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d4f12640 1 --2- 172.21.15.96:0/962794150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d0134e00 0x7f45d01351e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009d90 tx=0x7f45c001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc09058f788096b server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:49.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45cf7fe640 1 -- 172.21.15.96:0/962794150 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c00a5650 con 0x7f45d0134e00 2024-09-13T17:45:49.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45cf7fe640 1 -- 172.21.15.96:0/962794150 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c00a57b0 con 0x7f45d0134e00 2024-09-13T17:45:49.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45cf7fe640 1 -- 172.21.15.96:0/962794150 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c00a5a70 con 0x7f45d0134e00 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- 172.21.15.96:0/962794150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d0134e00 msgr2=0x7f45d01351e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 --2- 172.21.15.96:0/962794150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d0134e00 0x7f45d01351e0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f45c0009d90 tx=0x7f45c001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- 172.21.15.96:0/962794150 shutdown_connections 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 --2- 172.21.15.96:0/962794150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d0134e00 0x7f45d01351e0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- 172.21.15.96:0/962794150 >> 172.21.15.96:0/962794150 conn(0x7f45d0098a60 msgr2=0x7f45d009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- 172.21.15.96:0/962794150 shutdown_connections 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- 172.21.15.96:0/962794150 wait complete. 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 Processor -- start 2024-09-13T17:45:49.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.342+0000 7f45d7176640 1 -- start start 2024-09-13T17:45:49.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d7176640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d7176640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d011e130 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d4f12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d4f12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40044/0 (socket says 172.21.15.96:40044) 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d4f12640 1 -- 172.21.15.96:0/204814032 learned_addr learned my addr 172.21.15.96:0/204814032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d4f12640 1 -- 172.21.15.96:0/204814032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0009a70 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d4f12640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f45c00194f0 tx=0x7f45c0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c00b9580 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c00b96e0 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d0139df0 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f45c00b99a0 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d011c930 con 0x7f45d006f6d0 2024-09-13T17:45:49.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f45c00b9b00 con 0x7f45d006f6d0 2024-09-13T17:45:49.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d011d560 con 0x7f45d006f6d0 2024-09-13T17:45:49.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 0x7f45bc0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:49.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 3431+0+0 (secure 0 0 0) 0x7f45c00e2a80 con 0x7f45d006f6d0 2024-09-13T17:45:49.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cffff640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 0x7f45bc0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:49.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.346+0000 7f45cffff640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 0x7f45bc0379a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45c4000c10 tx=0x7f45c4007a50 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:49.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.350+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f45c00abb90 con 0x7f45d006f6d0 2024-09-13T17:45:49.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:49.518+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f45d0136ee0 con 0x7f45d006f6d0 2024-09-13T17:45:49.816 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:49.814+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:50.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.126+0000 7f45cd7fa640 1 -- 172.21.15.96:0/204814032 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7f45c00b2020 con 0x7f45d006f6d0 2024-09-13T17:45:50.129 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 msgr2=0x7f45bc0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 0x7f45bc0379a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f45c4000c10 tx=0x7f45c4007a50 comp rx=0 tx=0).stop 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 msgr2=0x7f45d0130d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f45c00194f0 tx=0x7f45c0019680 comp rx=0 tx=0).stop 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 shutdown_connections 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f45bc0354e0 0x7f45bc0379a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 --2- 172.21.15.96:0/204814032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f45d006f6d0 0x7f45d0130d60 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 >> 172.21.15.96:0/204814032 conn(0x7f45d0098a60 msgr2=0x7f45d0134200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 shutdown_connections 2024-09-13T17:45:50.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.130+0000 7f45d7176640 1 -- 172.21.15.96:0/204814032 wait complete. 2024-09-13T17:45:50.161 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-13T17:45:50.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b4155640 1 Processor -- start 2024-09-13T17:45:50.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b4155640 1 -- start start 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b4155640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f27ac09dc40 0x7f27ac136270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b4155640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac09ba60 con 0x7f27ac1367b0 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b4155640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac09cf60 con 0x7f27ac09dc40 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b1ef1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f27ac09dc40 0x7f27ac136270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b1ef1640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f27ac09dc40 0x7f27ac136270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40054/0 (socket says 172.21.15.96:40054) 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b16f0640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f27ac1367b0 0x7f27ac099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55714/0 (socket says 172.21.15.96:55714) 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b1ef1640 1 -- 172.21.15.96:0/1097596618 learned_addr learned my addr 172.21.15.96:0/1097596618 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b1ef1640 1 -- 172.21.15.96:0/1097596618 >> v1:172.21.15.96:6789/0 conn(0x7f27ac1367b0 legacy=0x7f27ac099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.290 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.286+0000 7f27b1ef1640 1 -- 172.21.15.96:0/1097596618 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac09c4e0 con 0x7f27ac09dc40 2024-09-13T17:45:50.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b1ef1640 1 --2- 172.21.15.96:0/1097596618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac136270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097f0 tx=0x7f27a801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8846ded1a883f463 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b0eef640 1 -- 172.21.15.96:0/1097596618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27a80ab530 con 0x7f27ac09dc40 2024-09-13T17:45:50.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b0eef640 1 -- 172.21.15.96:0/1097596618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a80a7050 con 0x7f27ac09dc40 2024-09-13T17:45:50.292 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b0eef640 1 -- 172.21.15.96:0/1097596618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27a80ab7e0 con 0x7f27ac09dc40 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- 172.21.15.96:0/1097596618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 msgr2=0x7f27ac136270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 --2- 172.21.15.96:0/1097596618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac136270 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f27a80097f0 tx=0x7f27a801c450 comp rx=0 tx=0).stop 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- 172.21.15.96:0/1097596618 shutdown_connections 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 --2- 172.21.15.96:0/1097596618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac136270 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- 172.21.15.96:0/1097596618 >> 172.21.15.96:0/1097596618 conn(0x7f27ac06f350 msgr2=0x7f27ac071770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- 172.21.15.96:0/1097596618 shutdown_connections 2024-09-13T17:45:50.293 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- 172.21.15.96:0/1097596618 wait complete. 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 Processor -- start 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- start start 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b4155640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0a34e0 con 0x7f27ac09dc40 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b1ef1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b1ef1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40056/0 (socket says 172.21.15.96:40056) 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.290+0000 7f27b1ef1640 1 -- 172.21.15.96:0/1921636126 learned_addr learned my addr 172.21.15.96:0/1921636126 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:50.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b1ef1640 1 -- 172.21.15.96:0/1921636126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a80094d0 con 0x7f27ac09dc40 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b1ef1640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f27a801a080 tx=0x7f27a80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27a80b4570 con 0x7f27ac09dc40 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a80b46d0 con 0x7f27ac09dc40 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac0a5430 con 0x7f27ac09dc40 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f27a80b4990 con 0x7f27ac09dc40 2024-09-13T17:45:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0a59b0 con 0x7f27ac09dc40 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0a57c0 con 0x7f27ac09dc40 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f27a80b2070 con 0x7f27ac09dc40 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 0x7f27900379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f27a80e1790 con 0x7f27ac09dc40 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b16f0640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 0x7f27900379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.297 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.294+0000 7f27b16f0640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 0x7f27900379a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f279c001600 tx=0x7f279c007a50 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.298+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f27a80c46c0 con 0x7f27ac09dc40 2024-09-13T17:45:50.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.470+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f27ac136270 con 0x7f27ac09dc40 2024-09-13T17:45:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27a27fc640 1 -- 172.21.15.96:0/1921636126 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7f27a80b80a0 con 0x7f27ac09dc40 2024-09-13T17:45:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 msgr2=0x7f27900379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 0x7f27900379a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f279c001600 tx=0x7f279c007a50 comp rx=0 tx=0).stop 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 msgr2=0x7f27ac1cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f27a801a080 tx=0x7f27a80191f0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 shutdown_connections 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f27900354e0 0x7f27900379a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 --2- 172.21.15.96:0/1921636126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ac09dc40 0x7f27ac1cfed0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 >> 172.21.15.96:0/1921636126 conn(0x7f27ac06f350 msgr2=0x7f27ac06b090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 shutdown_connections 2024-09-13T17:45:50.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.474+0000 7f27b4155640 1 -- 172.21.15.96:0/1921636126 wait complete. 2024-09-13T17:45:50.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster' 2024-09-13T17:45:50.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa2285d2640 1 Processor -- start 2024-09-13T17:45:50.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa2285d2640 1 -- start start 2024-09-13T17:45:50.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa2285d2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa22009e820 0x7fa22013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa2285d2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2200a68b0 con 0x7fa220134e70 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa2285d2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22009ddc0 con 0x7fa22009e820 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa225b6d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa22009e820 0x7fa22013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa225b6d640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fa22009e820 0x7fa22013ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40062/0 (socket says 172.21.15.96:40062) 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa225b6d640 1 -- 172.21.15.96:0/3035672687 learned_addr learned my addr 172.21.15.96:0/3035672687 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa22636e640 1 --1- 172.21.15.96:0/3035672687 >> v1:172.21.15.96:6789/0 conn(0x7fa220134e70 0x7fa220135250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55726/0 (socket says 172.21.15.96:55726) 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa225b6d640 1 -- 172.21.15.96:0/3035672687 >> v1:172.21.15.96:6789/0 conn(0x7fa220134e70 legacy=0x7fa220135250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.625 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.622+0000 7fa225b6d640 1 -- 172.21.15.96:0/3035672687 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220136970 con 0x7fa22009e820 2024-09-13T17:45:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa225b6d640 1 --2- 172.21.15.96:0/3035672687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22013ae50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0097f0 tx=0x7fa21c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4852f74b5c882d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22536c640 1 -- 172.21.15.96:0/3035672687 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa21c0ac540 con 0x7fa22009e820 2024-09-13T17:45:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22536c640 1 -- 172.21.15.96:0/3035672687 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c0ac6a0 con 0x7fa22009e820 2024-09-13T17:45:50.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22536c640 1 -- 172.21.15.96:0/3035672687 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa21c0ac960 con 0x7fa22009e820 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- 172.21.15.96:0/3035672687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 msgr2=0x7fa22013ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3035672687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22013ae50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0097f0 tx=0x7fa21c01c4e0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- 172.21.15.96:0/3035672687 shutdown_connections 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3035672687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22013ae50 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- 172.21.15.96:0/3035672687 >> 172.21.15.96:0/3035672687 conn(0x7fa220098a60 msgr2=0x7fa22009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- 172.21.15.96:0/3035672687 shutdown_connections 2024-09-13T17:45:50.628 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- 172.21.15.96:0/3035672687 wait complete. 2024-09-13T17:45:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 Processor -- start 2024-09-13T17:45:50.629 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- start start 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa2285d2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2200a50e0 con 0x7fa22009e820 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22636e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22636e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40064/0 (socket says 172.21.15.96:40064) 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22636e640 1 -- 172.21.15.96:0/3065830801 learned_addr learned my addr 172.21.15.96:0/3065830801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22636e640 1 -- 172.21.15.96:0/3065830801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21c0094d0 con 0x7fa22009e820 2024-09-13T17:45:50.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.626+0000 7fa22636e640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c410 tx=0x7fa21000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa210007410 con 0x7fa22009e820 2024-09-13T17:45:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa210007570 con 0x7fa22009e820 2024-09-13T17:45:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2200a3fd0 con 0x7fa22009e820 2024-09-13T17:45:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa2100047f0 con 0x7fa22009e820 2024-09-13T17:45:50.631 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa220071600 con 0x7fa22009e820 2024-09-13T17:45:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22011cb00 con 0x7fa22009e820 2024-09-13T17:45:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fa210004950 con 0x7fa22009e820 2024-09-13T17:45:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 0x7fa2040379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7fa2100477e0 con 0x7fa22009e820 2024-09-13T17:45:50.632 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa225b6d640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 0x7fa2040379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:50.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.630+0000 7fa225b6d640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 0x7fa2040379a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ae000 tx=0x7fa21c008040 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:50.636 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.634+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fa210004c00 con 0x7fa22009e820 2024-09-13T17:45:50.800 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:50.798+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:50.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:50.866+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster"]} v 0) v1 -- 0x7fa22006a1b0 con 0x7fa22009e820 2024-09-13T17:45:51.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa216ffd640 1 -- 172.21.15.96:0/3065830801 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_fstop.TestFSTop.test_fstop_non_existent_cluster"]}]=0 v226) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa21001be80 con 0x7fa22009e820 2024-09-13T17:45:51.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 msgr2=0x7fa2040379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 0x7fa2040379a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa21c0ae000 tx=0x7fa21c008040 comp rx=0 tx=0).stop 2024-09-13T17:45:51.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 msgr2=0x7fa22012f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fa21000c410 tx=0x7fa21000c8d0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 shutdown_connections 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fa2040354e0 0x7fa2040379a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 --2- 172.21.15.96:0/3065830801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa22009e820 0x7fa22012f5e0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 >> 172.21.15.96:0/3065830801 conn(0x7fa220098a60 msgr2=0x7fa220098e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 shutdown_connections 2024-09-13T17:45:51.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.138+0000 7fa2285d2640 1 -- 172.21.15.96:0/3065830801 wait complete. 2024-09-13T17:45:51.161 INFO:tasks.cephfs_test_runner:test_fstop_non_existent_cluster (tasks.cephfs.test_fstop.TestFSTop) ... ok 2024-09-13T17:45:51.162 INFO:tasks.cephfs_test_runner: 2024-09-13T17:45:51.162 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-13T17:45:51.162 INFO:tasks.cephfs_test_runner:Ran 4 tests in 201.919s 2024-09-13T17:45:51.162 INFO:tasks.cephfs_test_runner: 2024-09-13T17:45:51.162 INFO:tasks.cephfs_test_runner:OK 2024-09-13T17:45:51.162 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-13T17:45:51.171 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-13T17:45:51.179 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-13T17:45:51.179 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:51.179 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:51.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:45:51.184 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-13T17:45:51.184 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-13T17:45:51.231 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336468k,nr_inodes=4084117,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20533 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 2024-09-13T17:45:51.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:51.232 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-09-13T17:45:51.284 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 1) 2024-09-13T17:45:51.284 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-13T17:45:51.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:51.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-13T17:45:51.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:51.343 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-13T17:45:51.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-13T17:45:51.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-09-13T17:45:51.572 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T17:45:51.581 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T17:45:51.581 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:51.706 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.702+0000 7f098e607640 1 Processor -- start 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.702+0000 7f098e607640 1 -- start start 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.702+0000 7f098e607640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f098813af50 0x7f098813d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.702+0000 7f098e607640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0988138050 con 0x7f098809d690 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.702+0000 7f098e607640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098809c410 con 0x7f098813af50 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f097ffff640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f098813af50 0x7f098813d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f0987fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f098809d690 0x7f0988138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55730/0 (socket says 172.21.15.96:55730) 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f0987fff640 1 -- 172.21.15.96:0/2259789043 learned_addr learned my addr 172.21.15.96:0/2259789043 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f097ffff640 1 -- 172.21.15.96:0/2259789043 >> v1:172.21.15.96:6789/0 conn(0x7f098809d690 legacy=0x7f0988138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.707 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f097ffff640 1 -- 172.21.15.96:0/2259789043 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098809cba0 con 0x7f098813af50 2024-09-13T17:45:51.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f097ffff640 1 --2- 172.21.15.96:0/2259789043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098813af50 0x7f098813d340 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0970009740 tx=0x7f097001ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4d629be5d72683 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:51.709 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f09877fe640 1 -- 172.21.15.96:0/2259789043 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09700a8840 con 0x7f098813af50 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f09877fe640 1 -- 172.21.15.96:0/2259789043 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09700a89a0 con 0x7f098813af50 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f09877fe640 1 -- 172.21.15.96:0/2259789043 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09700a8c60 con 0x7f098813af50 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 -- 172.21.15.96:0/2259789043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098813af50 msgr2=0x7f098813d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 --2- 172.21.15.96:0/2259789043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098813af50 0x7f098813d340 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f0970009740 tx=0x7f097001ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 -- 172.21.15.96:0/2259789043 shutdown_connections 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 --2- 172.21.15.96:0/2259789043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098813af50 0x7f098813d340 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 -- 172.21.15.96:0/2259789043 >> 172.21.15.96:0/2259789043 conn(0x7f098806bc60 msgr2=0x7f098806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 -- 172.21.15.96:0/2259789043 shutdown_connections 2024-09-13T17:45:51.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.706+0000 7f098e607640 1 -- 172.21.15.96:0/2259789043 wait complete. 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 Processor -- start 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 -- start start 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09880a2d40 con 0x7f098809d690 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f0987fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f0987fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40082/0 (socket says 172.21.15.96:40082) 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f0987fff640 1 -- 172.21.15.96:0/3424680150 learned_addr learned my addr 172.21.15.96:0/3424680150 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f0987fff640 1 -- 172.21.15.96:0/3424680150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970009420 con 0x7f098809d690 2024-09-13T17:45:51.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f0987fff640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f097800c380 tx=0x7f097800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:51.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0978005c20 con 0x7f098809d690 2024-09-13T17:45:51.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978005d80 con 0x7f098809d690 2024-09-13T17:45:51.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09881372e0 con 0x7f098809d690 2024-09-13T17:45:51.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0978007200 con 0x7f098809d690 2024-09-13T17:45:51.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09880a5100 con 0x7f098809d690 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09880a4f10 con 0x7f098809d690 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f0978007360 con 0x7f098809d690 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 0x7f09680379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f0978046870 con 0x7f098809d690 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f097ffff640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 0x7f09680379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:51.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.710+0000 7f097ffff640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 0x7f09680379a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f09700b2000 tx=0x7f09700aa070 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:51.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.714+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f0978007610 con 0x7f098809d690 2024-09-13T17:45:51.756 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:51.754+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:51.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.878+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f098809b260 con 0x7f09680354e0 2024-09-13T17:45:51.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.878+0000 7f09857fa640 1 -- 172.21.15.96:0/3424680150 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13863 (secure 0 0 0) 0x7f09700a8c90 con 0x7f09680354e0 2024-09-13T17:45:51.881 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:51.882 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:51.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 msgr2=0x7f09680379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 0x7f09680379a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f09700b2000 tx=0x7f09700aa070 comp rx=0 tx=0).stop 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 msgr2=0x7f09881c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f097800c380 tx=0x7f097800c840 comp rx=0 tx=0).stop 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 shutdown_connections 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f09680354e0 0x7f09680379a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 --2- 172.21.15.96:0/3424680150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f098809d690 0x7f09881c9550 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 >> 172.21.15.96:0/3424680150 conn(0x7f098806bc60 msgr2=0x7f0988131300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 shutdown_connections 2024-09-13T17:45:51.884 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:51.882+0000 7f098e607640 1 -- 172.21.15.96:0/3424680150 wait complete. 2024-09-13T17:45:51.901 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-13T17:45:51.806531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":88380,"kb_used_data":3224,"kb_used_omap":4,"kb_used_meta":85115,"kb_avail":283027140,"statfs":{"total":289910292480,"available":289819791360,"internally_reserved":0,"allocated":3301376,"data_stored":2024105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":87158099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5135,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":-52,"num_write_kb":-22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004411"},"pg_stats":[{"pgid":"1.0","version":"74'3500","reported_seq":3763,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T17:45:51.788003+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:51.788003+0000","last_peered":"2024-09-13T17:45:51.788003+0000","last_clean":"2024-09-13T17:45:51.788003+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:51.788003+0000","last_undegraded":"2024-09-13T17:45:51.788003+0000","last_fullsized":"2024-09-13T17:45:51.788003+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":3500,"log_dups_size":0,"ondisk_log_size":3500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738417,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":696320,"data_stored":278697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66200000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738418,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:51.901 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8fb2e640 1 Processor -- start 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8fb2e640 1 -- start start 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8fb2e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a8809d690 0x7f8a88138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8fb2e640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a88138050 con 0x7f8a881387d0 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8fb2e640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8809c410 con 0x7f8a8809d690 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d8ca640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a8809d690 0x7f8a88138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d0c9640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f8a881387d0 0x7f8a88134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:55736/0 (socket says 172.21.15.96:55736) 2024-09-13T17:45:52.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d8ca640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f8a8809d690 0x7f8a88138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40098/0 (socket says 172.21.15.96:40098) 2024-09-13T17:45:52.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d0c9640 1 -- 172.21.15.96:0/2700116964 learned_addr learned my addr 172.21.15.96:0/2700116964 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:52.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d8ca640 1 -- 172.21.15.96:0/2700116964 >> v1:172.21.15.96:6789/0 conn(0x7f8a881387d0 legacy=0x7f8a88134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.022+0000 7f8a8d8ca640 1 -- 172.21.15.96:0/2700116964 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8809cba0 con 0x7f8a8809d690 2024-09-13T17:45:52.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8d8ca640 1 --2- 172.21.15.96:0/2700116964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a88138290 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009740 tx=0x7f8a8401ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781314e7793844d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8c8c8640 1 -- 172.21.15.96:0/2700116964 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a840a8840 con 0x7f8a8809d690 2024-09-13T17:45:52.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8c8c8640 1 -- 172.21.15.96:0/2700116964 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840a89a0 con 0x7f8a8809d690 2024-09-13T17:45:52.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8c8c8640 1 -- 172.21.15.96:0/2700116964 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a840a8c60 con 0x7f8a8809d690 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/2700116964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 msgr2=0x7f8a88138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/2700116964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a88138290 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f8a84009740 tx=0x7f8a8401ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/2700116964 shutdown_connections 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/2700116964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a88138290 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/2700116964 >> 172.21.15.96:0/2700116964 conn(0x7f8a8806bc60 msgr2=0x7f8a8806c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/2700116964 shutdown_connections 2024-09-13T17:45:52.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/2700116964 wait complete. 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 Processor -- start 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- start start 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.026+0000 7f8a8fb2e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880a5d10 con 0x7f8a8809d690 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d8ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d8ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40102/0 (socket says 172.21.15.96:40102) 2024-09-13T17:45:52.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d8ca640 1 -- 172.21.15.96:0/1079437716 learned_addr learned my addr 172.21.15.96:0/1079437716 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:52.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d8ca640 1 -- 172.21.15.96:0/1079437716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a84009420 con 0x7f8a8809d690 2024-09-13T17:45:52.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d8ca640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8a8401c840 tx=0x7f8a8401b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a840b0840 con 0x7f8a8809d690 2024-09-13T17:45:52.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a840b09a0 con 0x7f8a8809d690 2024-09-13T17:45:52.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a880a4470 con 0x7f8a8809d690 2024-09-13T17:45:52.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a840b0c60 con 0x7f8a8809d690 2024-09-13T17:45:52.032 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8809f490 con 0x7f8a8809d690 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f8a840b7040 con 0x7f8a8809d690 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8809bc00 con 0x7f8a8809d690 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 0x7f8a6c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f8a840e5230 con 0x7f8a8809d690 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d0c9640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 0x7f8a6c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.030+0000 7f8a8d0c9640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 0x7f8a6c0379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005890 tx=0x7f8a78007a90 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.034+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f8a8809bc00 con 0x7f8a8809d690 2024-09-13T17:45:52.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.214+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a8809c100 con 0x7f8a6c0354e0 2024-09-13T17:45:52.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a7e7fc640 1 -- 172.21.15.96:0/1079437716 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13863 (secure 0 0 0) 0x7f8a78001c00 con 0x7f8a6c0354e0 2024-09-13T17:45:52.219 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:52.220 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:52.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 msgr2=0x7f8a6c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 0x7f8a6c0379a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a78005890 tx=0x7f8a78007a90 comp rx=0 tx=0).stop 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 msgr2=0x7f8a880a6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f8a8401c840 tx=0x7f8a8401b9b0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 shutdown_connections 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f8a6c0354e0 0x7f8a6c0379a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 --2- 172.21.15.96:0/1079437716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a8809d690 0x7f8a880a6a60 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 >> 172.21.15.96:0/1079437716 conn(0x7f8a8806bc60 msgr2=0x7f8a88099460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 shutdown_connections 2024-09-13T17:45:52.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.218+0000 7f8a8fb2e640 1 -- 172.21.15.96:0/1079437716 wait complete. 2024-09-13T17:45:52.239 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-13T17:45:51.806531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":88380,"kb_used_data":3224,"kb_used_omap":4,"kb_used_meta":85115,"kb_avail":283027140,"statfs":{"total":289910292480,"available":289819791360,"internally_reserved":0,"allocated":3301376,"data_stored":2024105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":87158099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5135,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":-52,"num_write_kb":-22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004411"},"pg_stats":[{"pgid":"1.0","version":"74'3500","reported_seq":3763,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T17:45:51.788003+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:51.788003+0000","last_peered":"2024-09-13T17:45:51.788003+0000","last_clean":"2024-09-13T17:45:51.788003+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:51.788003+0000","last_undegraded":"2024-09-13T17:45:51.788003+0000","last_fullsized":"2024-09-13T17:45:51.788003+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":3500,"log_dups_size":0,"ondisk_log_size":3500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738417,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":696320,"data_stored":278697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66200000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738418,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:52.240 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T17:45:52.240 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:52.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb7ea2640 1 Processor -- start 2024-09-13T17:45:52.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb7ea2640 1 -- start start 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb7ea2640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efcb009d690 0x7efcb0138290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb7ea2640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb0138050 con 0x7efcb01387d0 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb7ea2640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb009c410 con 0x7efcb009d690 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb5c3e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efcb009d690 0x7efcb0138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb543d640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7efcb01387d0 0x7efcb0134650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36078/0 (socket says 172.21.15.96:36078) 2024-09-13T17:45:52.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb5c3e640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7efcb009d690 0x7efcb0138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38968/0 (socket says 172.21.15.96:38968) 2024-09-13T17:45:52.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb543d640 1 -- 172.21.15.96:0/3731660769 learned_addr learned my addr 172.21.15.96:0/3731660769 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:52.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb5c3e640 1 -- 172.21.15.96:0/3731660769 >> v1:172.21.15.96:6789/0 conn(0x7efcb01387d0 legacy=0x7efcb0134650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb5c3e640 1 -- 172.21.15.96:0/3731660769 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb009cba0 con 0x7efcb009d690 2024-09-13T17:45:52.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.362+0000 7efcb4c3c640 1 -- 172.21.15.96:0/3731660769 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621909749 0 0) 0x7efca0001120 con 0x7efcb01387d0 2024-09-13T17:45:52.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 --2- 172.21.15.96:0/3731660769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb0138290 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7efca8009740 tx=0x7efca801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9018526599342130 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb4c3c640 1 -- 172.21.15.96:0/3731660769 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca80a8840 con 0x7efcb009d690 2024-09-13T17:45:52.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb4c3c640 1 -- 172.21.15.96:0/3731660769 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80a89a0 con 0x7efcb009d690 2024-09-13T17:45:52.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb4c3c640 1 -- 172.21.15.96:0/3731660769 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca80a8c60 con 0x7efcb009d690 2024-09-13T17:45:52.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3731660769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 msgr2=0x7efcb0138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3731660769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb0138290 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7efca8009740 tx=0x7efca801ec10 comp rx=0 tx=0).stop 2024-09-13T17:45:52.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3731660769 shutdown_connections 2024-09-13T17:45:52.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3731660769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb0138290 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3731660769 >> 172.21.15.96:0/3731660769 conn(0x7efcb006bc60 msgr2=0x7efcb006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3731660769 shutdown_connections 2024-09-13T17:45:52.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3731660769 wait complete. 2024-09-13T17:45:52.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 Processor -- start 2024-09-13T17:45:52.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- start start 2024-09-13T17:45:52.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb00a4360 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38984/0 (socket says 172.21.15.96:38984) 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 -- 172.21.15.96:0/3565658605 learned_addr learned my addr 172.21.15.96:0/3565658605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 -- 172.21.15.96:0/3565658605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8009420 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb5c3e640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7efca801c840 tx=0x7efca801b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca80b0840 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80b09a0 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb00a7040 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efca80b0c60 con 0x7efcb009d690 2024-09-13T17:45:52.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.366+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb00a5db0 con 0x7efcb009d690 2024-09-13T17:45:52.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7efca80b7040 con 0x7efcb009d690 2024-09-13T17:45:52.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb009bc00 con 0x7efcb009d690 2024-09-13T17:45:52.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efca67fc640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 0x7efc940379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7efca80e5230 con 0x7efcb009d690 2024-09-13T17:45:52.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efcb543d640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 0x7efc940379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.373 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.370+0000 7efcb543d640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 0x7efc940379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efca0000c10 tx=0x7efca0001260 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.376 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.374+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7efcb009bc00 con 0x7efcb009d690 2024-09-13T17:45:52.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.546+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcb009c100 con 0x7efc940354e0 2024-09-13T17:45:52.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.546+0000 7efca67fc640 1 -- 172.21.15.96:0/3565658605 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13863 (secure 0 0 0) 0x7efca00058e0 con 0x7efc940354e0 2024-09-13T17:45:52.549 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:52.549 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:52.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 msgr2=0x7efc940379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 0x7efc940379a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efca0000c10 tx=0x7efca0001260 comp rx=0 tx=0).stop 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 msgr2=0x7efcb00a6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7efca801c840 tx=0x7efca801b9b0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 shutdown_connections 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7efc940354e0 0x7efc940379a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 --2- 172.21.15.96:0/3565658605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb009d690 0x7efcb00a6b00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 >> 172.21.15.96:0/3565658605 conn(0x7efcb006bc60 msgr2=0x7efcb0099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 shutdown_connections 2024-09-13T17:45:52.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.550+0000 7efcb7ea2640 1 -- 172.21.15.96:0/3565658605 wait complete. 2024-09-13T17:45:52.569 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-13T17:45:51.806531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":88380,"kb_used_data":3224,"kb_used_omap":4,"kb_used_meta":85115,"kb_avail":283027140,"statfs":{"total":289910292480,"available":289819791360,"internally_reserved":0,"allocated":3301376,"data_stored":2024105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":87158099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5135,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":-52,"num_write_kb":-22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004411"},"pg_stats":[{"pgid":"1.0","version":"74'3500","reported_seq":3763,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T17:45:51.788003+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:51.788003+0000","last_peered":"2024-09-13T17:45:51.788003+0000","last_clean":"2024-09-13T17:45:51.788003+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:51.788003+0000","last_undegraded":"2024-09-13T17:45:51.788003+0000","last_fullsized":"2024-09-13T17:45:51.788003+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":3500,"log_dups_size":0,"ondisk_log_size":3500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738417,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":696320,"data_stored":278697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66200000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738418,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:52.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T17:45:52.656 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:52.654+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:52.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95794f4640 1 Processor -- start 2024-09-13T17:45:52.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95794f4640 1 -- start start 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95794f4640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f95740a4900 0x7f957409e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95794f4640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9574071740 con 0x7f95740a4ce0 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95794f4640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957409e250 con 0x7f95740a4900 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f95740a4900 0x7f957409e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f95740a4900 0x7f957409e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38994/0 (socket says 172.21.15.96:38994) 2024-09-13T17:45:52.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f956bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f95740a4ce0 0x7f95740a50f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36080/0 (socket says 172.21.15.96:36080) 2024-09-13T17:45:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 -- 172.21.15.96:0/3887467035 learned_addr learned my addr 172.21.15.96:0/3887467035 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 -- 172.21.15.96:0/3887467035 >> v1:172.21.15.96:6789/0 conn(0x7f95740a4ce0 legacy=0x7f95740a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 -- 172.21.15.96:0/3887467035 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740a5c70 con 0x7f95740a4900 2024-09-13T17:45:52.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f95727fc640 1 -- 172.21.15.96:0/3887467035 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3319940078 0 0) 0x7f9574071740 con 0x7f95740a4ce0 2024-09-13T17:45:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.690+0000 7f9572ffd640 1 --2- 172.21.15.96:0/3887467035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957409e9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f95640097f0 tx=0x7f956401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1a13b563a1fc94 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95727fc640 1 -- 172.21.15.96:0/3887467035 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95640ab530 con 0x7f95740a4900 2024-09-13T17:45:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95727fc640 1 -- 172.21.15.96:0/3887467035 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95640a7050 con 0x7f95740a4900 2024-09-13T17:45:52.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95727fc640 1 -- 172.21.15.96:0/3887467035 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95640ab7e0 con 0x7f95740a4900 2024-09-13T17:45:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3887467035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 msgr2=0x7f957409e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 --2- 172.21.15.96:0/3887467035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957409e9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f95640097f0 tx=0x7f956401c450 comp rx=0 tx=0).stop 2024-09-13T17:45:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3887467035 shutdown_connections 2024-09-13T17:45:52.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 --2- 172.21.15.96:0/3887467035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957409e9a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3887467035 >> 172.21.15.96:0/3887467035 conn(0x7f957406a150 msgr2=0x7f957406a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3887467035 shutdown_connections 2024-09-13T17:45:52.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3887467035 wait complete. 2024-09-13T17:45:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 Processor -- start 2024-09-13T17:45:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- start start 2024-09-13T17:45:52.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9574071170 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f9572ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f9572ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39010/0 (socket says 172.21.15.96:39010) 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f9572ffd640 1 -- 172.21.15.96:0/3132970200 learned_addr learned my addr 172.21.15.96:0/3132970200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f9572ffd640 1 -- 172.21.15.96:0/3132970200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95640094d0 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f9572ffd640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9564019060 tx=0x7f956401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95640abea0 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9564004020 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957413c530 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95640042c0 con 0x7f95740a4900 2024-09-13T17:45:52.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.694+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957411de80 con 0x7f95740a4900 2024-09-13T17:45:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f95640b2040 con 0x7f95740a4900 2024-09-13T17:45:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957411d270 con 0x7f95740a4900 2024-09-13T17:45:52.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f956b7fe640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 0x7f95580379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f95640e1660 con 0x7f95740a4900 2024-09-13T17:45:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f956bfff640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 0x7f95580379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:52.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.698+0000 7f956bfff640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 0x7f95580379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f955c007fd0 tx=0x7f955c007920 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:52.704 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.702+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f95640c6300 con 0x7f95740a4900 2024-09-13T17:45:52.874 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.870+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9574073570 con 0x7f95740a4900 2024-09-13T17:45:52.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f956b7fe640 1 -- 172.21.15.96:0/3132970200 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+7354 (secure 0 0 0) 0x7f95640b0020 con 0x7f95740a4900 2024-09-13T17:45:52.875 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:52.875 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":74,"fsid":"a54aa7b1-da4c-43a3-b94f-1607e77cbcc4","created":"2024-09-13T17:41:41.090153+0000","modified":"2024-09-13T17:45:50.122954+0000","last_up_change":"2024-09-13T17:41:50.588222+0000","last_in_change":"2024-09-13T17:41:41.949857+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-13T17:41:51.865891+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"554ada03-d5e2-4466-95ee-96adb96f409f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6809","nonce":749556932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6811","nonce":749556932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6815","nonce":749556932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":749556932},{"type":"v1","addr":"172.21.15.96:6813","nonce":749556932}]},"public_addr":"172.21.15.96:6809/749556932","cluster_addr":"172.21.15.96:6811/749556932","heartbeat_back_addr":"172.21.15.96:6815/749556932","heartbeat_front_addr":"172.21.15.96:6813/749556932","state":["exists","up"]},{"osd":1,"uuid":"97e27884-eab0-4aaa-b570-7baf268781e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6817","nonce":3848754233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6819","nonce":3848754233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6823","nonce":3848754233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":3848754233},{"type":"v1","addr":"172.21.15.96:6821","nonce":3848754233}]},"public_addr":"172.21.15.96:6817/3848754233","cluster_addr":"172.21.15.96:6819/3848754233","heartbeat_back_addr":"172.21.15.96:6823/3848754233","heartbeat_front_addr":"172.21.15.96:6821/3848754233","state":["exists","up"]},{"osd":2,"uuid":"5195b731-8e00-4da1-a5ad-ff98b219f282","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6801","nonce":654054578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6803","nonce":654054578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6807","nonce":654054578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":654054578},{"type":"v1","addr":"172.21.15.96:6805","nonce":654054578}]},"public_addr":"172.21.15.96:6801/654054578","cluster_addr":"172.21.15.96:6803/654054578","heartbeat_back_addr":"172.21.15.96:6807/654054578","heartbeat_front_addr":"172.21.15.96:6805/654054578","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.219129+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.543291+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T17:41:47.292297+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:6829/1571674225":"2024-09-14T17:45:47.113743+0000","172.21.15.96:6828/1571674225":"2024-09-14T17:45:47.113743+0000","172.21.15.96:0/2416365020":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/3625984202":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6825/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:6824/3517445048":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1018463101":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/928338259":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1348634400":"2024-09-14T17:45:42.332380+0000","172.21.15.96:0/1206016791":"2024-09-13T18:45:38.078633+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T17:45:52.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 msgr2=0x7f95580379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 0x7f95580379a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f955c007fd0 tx=0x7f955c007920 comp rx=0 tx=0).stop 2024-09-13T17:45:52.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 msgr2=0x7f957411e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f9564019060 tx=0x7f956401ac90 comp rx=0 tx=0).stop 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 shutdown_connections 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f95580354e0 0x7f95580379a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 --2- 172.21.15.96:0/3132970200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95740a4900 0x7f957411e600 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 >> 172.21.15.96:0/3132970200 conn(0x7f957406a150 msgr2=0x7f95741386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 shutdown_connections 2024-09-13T17:45:52.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:52.874+0000 7f95794f4640 1 -- 172.21.15.96:0/3132970200 wait complete. 2024-09-13T17:45:53.661 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:53.658+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:53.895 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T17:45:53.895 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:45:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc121cc3640 1 Processor -- start 2024-09-13T17:45:54.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc121cc3640 1 -- start start 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc121cc3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc11c1319d0 0x7fc11c131db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc121cc3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11c0a0f80 con 0x7fc11c0a6610 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc121cc3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11c06ef50 con 0x7fc11c1319d0 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc11b7fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fc11c1319d0 0x7fc11c131db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc11affd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fc11c0a6610 0x7fc11c134a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36096/0 (socket says 172.21.15.96:36096) 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc11affd640 1 -- 172.21.15.96:0/2255628120 learned_addr learned my addr 172.21.15.96:0/2255628120 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:54.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc11b7fe640 1 -- 172.21.15.96:0/2255628120 >> v1:172.21.15.96:6789/0 conn(0x7fc11c0a6610 legacy=0x7fc11c134a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.034+0000 7fc11b7fe640 1 -- 172.21.15.96:0/2255628120 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11c06f710 con 0x7fc11c1319d0 2024-09-13T17:45:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc11b7fe640 1 --2- 172.21.15.96:0/2255628120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c131db0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc11c06f090 tx=0x7fc10c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4a7b7614b8c73c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc11a7fc640 1 -- 172.21.15.96:0/2255628120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c0ab530 con 0x7fc11c1319d0 2024-09-13T17:45:54.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc11a7fc640 1 -- 172.21.15.96:0/2255628120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c0a7050 con 0x7fc11c1319d0 2024-09-13T17:45:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc11a7fc640 1 -- 172.21.15.96:0/2255628120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c0ab7e0 con 0x7fc11c1319d0 2024-09-13T17:45:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- 172.21.15.96:0/2255628120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 msgr2=0x7fc11c131db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 --2- 172.21.15.96:0/2255628120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c131db0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fc11c06f090 tx=0x7fc10c01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- 172.21.15.96:0/2255628120 shutdown_connections 2024-09-13T17:45:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 --2- 172.21.15.96:0/2255628120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c131db0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- 172.21.15.96:0/2255628120 >> 172.21.15.96:0/2255628120 conn(0x7fc11c06a150 msgr2=0x7fc11c06a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- 172.21.15.96:0/2255628120 shutdown_connections 2024-09-13T17:45:54.042 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- 172.21.15.96:0/2255628120 wait complete. 2024-09-13T17:45:54.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 Processor -- start 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.038+0000 7fc121cc3640 1 -- start start 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc121cc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc121cc3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c09b850 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39026/0 (socket says 172.21.15.96:39026) 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11b7fe640 1 -- 172.21.15.96:0/3884204893 learned_addr learned my addr 172.21.15.96:0/3884204893 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11b7fe640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c009420 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11b7fe640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc10c019fc0 tx=0x7fc10c01abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c0b7450 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10c0b75b0 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c09c460 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc10c0b7890 con 0x7fc11c1319d0 2024-09-13T17:45:54.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1394f0 con 0x7fc11c1319d0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc11c13cdc0 con 0x7fc11c1319d0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fc10c0b79f0 con 0x7fc11c1319d0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 0x7fc1080379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7fc10c0e2090 con 0x7fc11c1319d0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 0x7fc10803c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11affd640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 0x7fc1080379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc10803ce20 con 0x7fc10803a310 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc10c0e4700 con 0x7fc11c1319d0 2024-09-13T17:45:54.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11bfff640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 0x7fc10803c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11affd640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 0x7fc1080379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc110001610 tx=0x7fc110007ad0 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc11bfff640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 0x7fc10803c750 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.042+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== osd.0 v2:172.21.15.96:6808/749556932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fc10803ce20 con 0x7fc10803a310 2024-09-13T17:45:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 --> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc11c131db0 con 0x7fc10803a310 2024-09-13T17:45:54.059 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc103fff640 1 -- 172.21.15.96:0/3884204893 <== osd.0 v2:172.21.15.96:6808/749556932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc11c131db0 con 0x7fc10803a310 2024-09-13T17:45:54.059 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:54.060 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:45:54.060 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 msgr2=0x7fc10803c750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 0x7fc10803c750 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 msgr2=0x7fc1080379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 0x7fc1080379a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc110001610 tx=0x7fc110007ad0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 msgr2=0x7fc11c137d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fc10c019fc0 tx=0x7fc10c01abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 shutdown_connections 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6808/749556932,v1:172.21.15.96:6809/749556932] conn(0x7fc10803a310 0x7fc10803c750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fc1080354e0 0x7fc1080379a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 --2- 172.21.15.96:0/3884204893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc11c1319d0 0x7fc11c137d50 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 >> 172.21.15.96:0/3884204893 conn(0x7fc11c06a150 msgr2=0x7fc11c1335f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 shutdown_connections 2024-09-13T17:45:54.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.058+0000 7fc121cc3640 1 -- 172.21.15.96:0/3884204893 wait complete. 2024-09-13T17:45:54.076 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T17:45:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11e4b24640 1 Processor -- start 2024-09-13T17:45:54.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11e4b24640 1 -- start start 2024-09-13T17:45:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11e4b24640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11e013af50 0x7f11e013d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11e4b24640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e0138050 con 0x7f11e009d690 2024-09-13T17:45:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11e4b24640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e009c410 con 0x7f11e013af50 2024-09-13T17:45:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11ddd9b640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f11e013af50 0x7f11e013d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11de59c640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f11e009d690 0x7f11e0138290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36110/0 (socket says 172.21.15.96:36110) 2024-09-13T17:45:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11de59c640 1 -- 172.21.15.96:0/3946326766 learned_addr learned my addr 172.21.15.96:0/3946326766 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11ddd9b640 1 -- 172.21.15.96:0/3946326766 >> v1:172.21.15.96:6789/0 conn(0x7f11e009d690 legacy=0x7f11e0138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11ddd9b640 1 -- 172.21.15.96:0/3946326766 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e009cba0 con 0x7f11e013af50 2024-09-13T17:45:54.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11dd59a640 1 -- 172.21.15.96:0/3946326766 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396591515 0 0) 0x7f11d8001120 con 0x7f11e009d690 2024-09-13T17:45:54.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.190+0000 7f11ddd9b640 1 --2- 172.21.15.96:0/3946326766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e013af50 0x7f11e013d340 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009d90 tx=0x7f11d401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfaf60dec760214 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11dd59a640 1 -- 172.21.15.96:0/3946326766 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d40a8e10 con 0x7f11e013af50 2024-09-13T17:45:54.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11dd59a640 1 -- 172.21.15.96:0/3946326766 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d40b0440 con 0x7f11e013af50 2024-09-13T17:45:54.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11dd59a640 1 -- 172.21.15.96:0/3946326766 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d40b0700 con 0x7f11e013af50 2024-09-13T17:45:54.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/3946326766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e013af50 msgr2=0x7f11e013d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 --2- 172.21.15.96:0/3946326766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e013af50 0x7f11e013d340 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009d90 tx=0x7f11d401f150 comp rx=0 tx=0).stop 2024-09-13T17:45:54.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/3946326766 shutdown_connections 2024-09-13T17:45:54.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 --2- 172.21.15.96:0/3946326766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e013af50 0x7f11e013d340 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/3946326766 >> 172.21.15.96:0/3946326766 conn(0x7f11e006bc60 msgr2=0x7f11e006c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/3946326766 shutdown_connections 2024-09-13T17:45:54.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/3946326766 wait complete. 2024-09-13T17:45:54.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 Processor -- start 2024-09-13T17:45:54.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- start start 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00a6f20 con 0x7f11e009d690 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11de59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39044/0 (socket says 172.21.15.96:39044) 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11de59c640 1 -- 172.21.15.96:0/426608272 learned_addr learned my addr 172.21.15.96:0/426608272 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11de59c640 1 -- 172.21.15.96:0/426608272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4009a70 con 0x7f11e009d690 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11de59c640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f11d800c390 tx=0x7f11d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d8007410 con 0x7f11e009d690 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d8007590 con 0x7f11e009d690 2024-09-13T17:45:54.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11d80047f0 con 0x7f11e009d690 2024-09-13T17:45:54.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e00a4cb0 con 0x7f11e009d690 2024-09-13T17:45:54.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.194+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e00a4ee0 con 0x7f11e009d690 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e00a4430 con 0x7f11e009d690 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f11d8004950 con 0x7f11e009d690 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11ceffd640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 0x7f11c00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f11d8047190 con 0x7f11e009d690 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11ddd9b640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 0x7f11c00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.198+0000 7f11ddd9b640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 0x7f11c00379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11d40b2000 tx=0x7f11d40aa070 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.202+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f11e00a4430 con 0x7f11e009d690 2024-09-13T17:45:54.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f11e009c100 con 0x7f11c00354e0 2024-09-13T17:45:54.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11ceffd640 1 -- 172.21.15.96:0/426608272 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f11d40c0090 con 0x7f11c00354e0 2024-09-13T17:45:54.376 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 msgr2=0x7f11c00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 0x7f11c00379a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11d40b2000 tx=0x7f11d40aa070 comp rx=0 tx=0).stop 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 msgr2=0x7f11e00a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f11d800c390 tx=0x7f11d800c850 comp rx=0 tx=0).stop 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 shutdown_connections 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f11c00354e0 0x7f11c00379a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 --2- 172.21.15.96:0/426608272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11e009d690 0x7f11e00a69e0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 >> 172.21.15.96:0/426608272 conn(0x7f11e006bc60 msgr2=0x7f11e0139dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 shutdown_connections 2024-09-13T17:45:54.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.374+0000 7f11e4b24640 1 -- 172.21.15.96:0/426608272 wait complete. 2024-09-13T17:45:54.395 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T17:45:54.396 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:45:54.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d92258640 1 Processor -- start 2024-09-13T17:45:54.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d92258640 1 -- start start 2024-09-13T17:45:54.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d92258640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5d8c138290 0x7f5d8c09fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d92258640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d8c09e520 con 0x7f5d8c138670 2024-09-13T17:45:54.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d92258640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d8c0aabe0 con 0x7f5d8c138290 2024-09-13T17:45:54.516 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d83fff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5d8c138670 0x7f5d8c09ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36126/0 (socket says 172.21.15.96:36126) 2024-09-13T17:45:54.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d83fff640 1 -- 172.21.15.96:0/1874565998 learned_addr learned my addr 172.21.15.96:0/1874565998 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:54.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d8b7fe640 1 --2- 172.21.15.96:0/1874565998 >> v2:172.21.15.96:3300/0 conn(0x7f5d8c138290 0x7f5d8c09fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d8b7fe640 1 -- 172.21.15.96:0/1874565998 >> v1:172.21.15.96:6789/0 conn(0x7f5d8c138670 legacy=0x7f5d8c09ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d8b7fe640 1 -- 172.21.15.96:0/1874565998 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c09d610 con 0x7f5d8c138290 2024-09-13T17:45:54.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.514+0000 7f5d8affd640 1 -- 172.21.15.96:0/1874565998 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964659506 0 0) 0x7f5d8c09e520 con 0x7f5d8c138670 2024-09-13T17:45:54.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 --2- 172.21.15.96:0/1874565998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c09fd20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009740 tx=0x7f5d7401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d6bea47ba50b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8affd640 1 -- 172.21.15.96:0/1874565998 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d740ab530 con 0x7f5d8c138290 2024-09-13T17:45:54.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8affd640 1 -- 172.21.15.96:0/1874565998 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740a7050 con 0x7f5d8c138290 2024-09-13T17:45:54.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8affd640 1 -- 172.21.15.96:0/1874565998 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d740ab7e0 con 0x7f5d8c138290 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1874565998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 msgr2=0x7f5d8c09fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 --2- 172.21.15.96:0/1874565998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c09fd20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009740 tx=0x7f5d7401c390 comp rx=0 tx=0).stop 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1874565998 shutdown_connections 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 --2- 172.21.15.96:0/1874565998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c09fd20 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1874565998 >> 172.21.15.96:0/1874565998 conn(0x7f5d8c06c060 msgr2=0x7f5d8c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1874565998 shutdown_connections 2024-09-13T17:45:54.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1874565998 wait complete. 2024-09-13T17:45:54.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 Processor -- start 2024-09-13T17:45:54.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- start start 2024-09-13T17:45:54.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c0a46c0 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39058/0 (socket says 172.21.15.96:39058) 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 -- 172.21.15.96:0/1203802414 learned_addr learned my addr 172.21.15.96:0/1203802414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d74009420 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d8b7fe640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5d740ae040 tx=0x7f5d7401abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d740b7580 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740b76e0 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d740b79c0 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d8c13d2b0 con 0x7f5d8c138290 2024-09-13T17:45:54.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.518+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d8c1cc370 con 0x7f5d8c138290 2024-09-13T17:45:54.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d8c09e750 con 0x7f5d8c138290 2024-09-13T17:45:54.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5d740b7b20 con 0x7f5d8c138290 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 0x7f5d6c0379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f5d740e2770 con 0x7f5d8c138290 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 0x7f5d6c03c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d6c03cdd0 con 0x7f5d6c03a2c0 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d740e2b70 con 0x7f5d8c138290 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d83fff640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 0x7f5d6c0379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d8bfff640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 0x7f5d6c03c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d83fff640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 0x7f5d6c0379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c007fd0 tx=0x7f5d7c007920 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.524 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d8bfff640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 0x7f5d6c03c700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.525 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.522+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== osd.1 v2:172.21.15.96:6816/3848754233 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7f5d6c03cdd0 con 0x7f5d6c03a2c0 2024-09-13T17:45:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.534+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 --> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5d8c0a5f10 con 0x7f5d6c03a2c0 2024-09-13T17:45:54.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.534+0000 7f5d88ff9640 1 -- 172.21.15.96:0/1203802414 <== osd.1 v2:172.21.15.96:6816/3848754233 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5d8c0a5f10 con 0x7f5d6c03a2c0 2024-09-13T17:45:54.538 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:54.538 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:45:54.538 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.534+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 msgr2=0x7f5d6c03c700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.534+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 0x7f5d6c03c700 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 msgr2=0x7f5d6c0379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 0x7f5d6c0379a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c007fd0 tx=0x7f5d7c007920 comp rx=0 tx=0).stop 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 msgr2=0x7f5d8c1cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f5d740ae040 tx=0x7f5d7401abd0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 shutdown_connections 2024-09-13T17:45:54.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5d6c0354e0 0x7f5d6c0379a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:6816/3848754233,v1:172.21.15.96:6817/3848754233] conn(0x7f5d6c03a2c0 0x7f5d6c03c700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 --2- 172.21.15.96:0/1203802414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5d8c138290 0x7f5d8c1cc680 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 >> 172.21.15.96:0/1203802414 conn(0x7f5d8c06c060 msgr2=0x7f5d8c1373d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 shutdown_connections 2024-09-13T17:45:54.540 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.538+0000 7f5d92258640 1 -- 172.21.15.96:0/1203802414 wait complete. 2024-09-13T17:45:54.555 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T17:45:54.640 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:54.638+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:54.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.666+0000 7f6f286cc640 1 Processor -- start 2024-09-13T17:45:54.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.666+0000 7f6f286cc640 1 -- start start 2024-09-13T17:45:54.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6f2009c160 0x7f6f2013ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f20135b60 con 0x7f6f20135f70 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f200720e0 con 0x7f6f2009c160 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25c67640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f6f2009c160 0x7f6f2013ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f26468640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f6f20135f70 0x7f6f20136350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36130/0 (socket says 172.21.15.96:36130) 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f26468640 1 -- 172.21.15.96:0/1435682788 learned_addr learned my addr 172.21.15.96:0/1435682788 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25c67640 1 -- 172.21.15.96:0/1435682788 >> v1:172.21.15.96:6789/0 conn(0x7f6f20135f70 legacy=0x7f6f20136350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25466640 1 -- 172.21.15.96:0/1435682788 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088660314 0 0) 0x7f6f20135b60 con 0x7f6f20135f70 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25c67640 1 -- 172.21.15.96:0/1435682788 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2009cf60 con 0x7f6f2009c160 2024-09-13T17:45:54.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25c67640 1 --2- 172.21.15.96:0/1435682788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f2013ad70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009740 tx=0x7f6f1001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa91b8397f688047 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25466640 1 -- 172.21.15.96:0/1435682788 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f100ab530 con 0x7f6f2009c160 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25466640 1 -- 172.21.15.96:0/1435682788 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f100a7050 con 0x7f6f2009c160 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f25466640 1 -- 172.21.15.96:0/1435682788 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f100ab7e0 con 0x7f6f2009c160 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- 172.21.15.96:0/1435682788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 msgr2=0x7f6f2013ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 --2- 172.21.15.96:0/1435682788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f2013ad70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009740 tx=0x7f6f1001c390 comp rx=0 tx=0).stop 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- 172.21.15.96:0/1435682788 shutdown_connections 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 --2- 172.21.15.96:0/1435682788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f2013ad70 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- 172.21.15.96:0/1435682788 >> 172.21.15.96:0/1435682788 conn(0x7f6f2006ec70 msgr2=0x7f6f20071090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- 172.21.15.96:0/1435682788 shutdown_connections 2024-09-13T17:45:54.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.670+0000 7f6f286cc640 1 -- 172.21.15.96:0/1435682788 wait complete. 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 Processor -- start 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 -- start start 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2013a770 con 0x7f6f2009c160 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f26468640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f26468640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39084/0 (socket says 172.21.15.96:39084) 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f26468640 1 -- 172.21.15.96:0/2234564888 learned_addr learned my addr 172.21.15.96:0/2234564888 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f26468640 1 -- 172.21.15.96:0/2234564888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f10009420 con 0x7f6f2009c160 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f26468640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c420 tx=0x7f6f1c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.679 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f1c0074d0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f1c007630 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f200a00f0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f1c004960 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f20130fb0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f200008e0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f6f1c004ac0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 0x7f6f040379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f6f1c046af0 con 0x7f6f2009c160 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f25c67640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 0x7f6f040379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.674+0000 7f6f25c67640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 0x7f6f040379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ad000 tx=0x7f6f10008040 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.682+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6f1c01ab20 con 0x7f6f2009c160 2024-09-13T17:45:54.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.842+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f20139ad0 con 0x7f6f040354e0 2024-09-13T17:45:54.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.842+0000 7f6f16ffd640 1 -- 172.21.15.96:0/2234564888 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6f20139ad0 con 0x7f6f040354e0 2024-09-13T17:45:54.845 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 msgr2=0x7f6f040379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 0x7f6f040379a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6f100ad000 tx=0x7f6f10008040 comp rx=0 tx=0).stop 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 msgr2=0x7f6f20138140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c420 tx=0x7f6f1c00c8e0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 shutdown_connections 2024-09-13T17:45:54.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6f040354e0 0x7f6f040379a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 --2- 172.21.15.96:0/2234564888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f2009c160 0x7f6f20138140 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 >> 172.21.15.96:0/2234564888 conn(0x7f6f2006ec70 msgr2=0x7f6f20134fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 shutdown_connections 2024-09-13T17:45:54.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.846+0000 7f6f286cc640 1 -- 172.21.15.96:0/2234564888 wait complete. 2024-09-13T17:45:54.863 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T17:45:54.863 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T17:45:54.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc5dde3640 1 Processor -- start 2024-09-13T17:45:54.989 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc5dde3640 1 -- start start 2024-09-13T17:45:54.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc5dde3640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbc58135e80 0x7fbc5809e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc5dde3640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5809e0d0 con 0x7fbc58136260 2024-09-13T17:45:54.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc5dde3640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5809ef50 con 0x7fbc58135e80 2024-09-13T17:45:54.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc577fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fbc58135e80 0x7fbc5809e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc56ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fbc58136260 0x7fbc5806dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36132/0 (socket says 172.21.15.96:36132) 2024-09-13T17:45:54.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc56ffd640 1 -- 172.21.15.96:0/802748984 learned_addr learned my addr 172.21.15.96:0/802748984 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:54.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc577fe640 1 -- 172.21.15.96:0/802748984 >> v1:172.21.15.96:6789/0 conn(0x7fbc58136260 legacy=0x7fbc5806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc577fe640 1 -- 172.21.15.96:0/802748984 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5809bf40 con 0x7fbc58135e80 2024-09-13T17:45:54.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.986+0000 7fbc567fc640 1 -- 172.21.15.96:0/802748984 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478714926 0 0) 0x7fbc5809e0d0 con 0x7fbc58136260 2024-09-13T17:45:54.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc577fe640 1 --2- 172.21.15.96:0/802748984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc5809e890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009740 tx=0x7fbc4c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12b5cfee724f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc567fc640 1 -- 172.21.15.96:0/802748984 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c0ab530 con 0x7fbc58135e80 2024-09-13T17:45:54.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc567fc640 1 -- 172.21.15.96:0/802748984 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0a7050 con 0x7fbc58135e80 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc567fc640 1 -- 172.21.15.96:0/802748984 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c0ab7e0 con 0x7fbc58135e80 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- 172.21.15.96:0/802748984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 msgr2=0x7fbc5809e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/802748984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc5809e890 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c009740 tx=0x7fbc4c01c420 comp rx=0 tx=0).stop 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- 172.21.15.96:0/802748984 shutdown_connections 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/802748984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc5809e890 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- 172.21.15.96:0/802748984 >> 172.21.15.96:0/802748984 conn(0x7fbc58098a60 msgr2=0x7fbc5809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- 172.21.15.96:0/802748984 shutdown_connections 2024-09-13T17:45:54.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- 172.21.15.96:0/802748984 wait complete. 2024-09-13T17:45:54.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 Processor -- start 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.990+0000 7fbc5dde3640 1 -- start start 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc5dde3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc5dde3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc58071200 con 0x7fbc58135e80 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39110/0 (socket says 172.21.15.96:39110) 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc577fe640 1 -- 172.21.15.96:0/2975117467 learned_addr learned my addr 172.21.15.96:0/2975117467 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc577fe640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c009420 con 0x7fbc58135e80 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc577fe640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0ae040 tx=0x7fbc4c0191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c0b4540 con 0x7fbc58135e80 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4c0b46a0 con 0x7fbc58135e80 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc58071510 con 0x7fbc58135e80 2024-09-13T17:45:54.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbc4c0b4960 con 0x7fbc58135e80 2024-09-13T17:45:54.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc58070290 con 0x7fbc58135e80 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbc5813de00 con 0x7fbc58135e80 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7fbc4c0b2040 con 0x7fbc58135e80 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 0x7fbc44037950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7fbc4c0e1d00 con 0x7fbc58135e80 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 0x7fbc4403c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc56ffd640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 0x7fbc44037950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbc4403cd80 con 0x7fbc4403a270 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc4c0e43f0 con 0x7fbc58135e80 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc57fff640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 0x7fbc4403c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:54.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc56ffd640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 0x7fbc44037950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007fd0 tx=0x7fbc48007920 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc57fff640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 0x7fbc4403c6b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:54.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:54.994+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== osd.2 v2:172.21.15.96:6800/654054578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26118 (crc 0 0 0) 0x7fbc4403cd80 con 0x7fbc4403a270 2024-09-13T17:45:55.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 --> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbc58071340 con 0x7fbc4403a270 2024-09-13T17:45:55.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc3ffff640 1 -- 172.21.15.96:0/2975117467 <== osd.2 v2:172.21.15.96:6800/654054578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbc58071340 con 0x7fbc4403a270 2024-09-13T17:45:55.011 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-13T17:45:55.011 INFO:teuthology.orchestra.run.smithi096.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T17:45:55.011 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-13T17:45:55.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 msgr2=0x7fbc4403c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 0x7fbc4403c6b0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 msgr2=0x7fbc44037950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 0x7fbc44037950 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc48007fd0 tx=0x7fbc48007920 comp rx=0 tx=0).stop 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 msgr2=0x7fbc581cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c0ae040 tx=0x7fbc4c0191c0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 shutdown_connections 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6800/654054578,v1:172.21.15.96:6801/654054578] conn(0x7fbc4403a270 0x7fbc4403c6b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fbc44035490 0x7fbc44037950 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 --2- 172.21.15.96:0/2975117467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc58135e80 0x7fbc581cfb50 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 >> 172.21.15.96:0/2975117467 conn(0x7fbc58098a60 msgr2=0x7fbc5809a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 shutdown_connections 2024-09-13T17:45:55.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.010+0000 7fbc5dde3640 1 -- 172.21.15.96:0/2975117467 wait complete. 2024-09-13T17:45:55.026 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T17:45:55.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e27ac0640 1 Processor -- start 2024-09-13T17:45:55.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e27ac0640 1 -- start start 2024-09-13T17:45:55.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e27ac0640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e20134670 0x7f5e20134a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e27ac0640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e20071670 con 0x7f5e2006e5c0 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e27ac0640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e20071360 con 0x7f5e20134670 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2585c640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f5e20134670 0x7f5e20134a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2505b640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f5e2006e5c0 0x7f5e2006ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36142/0 (socket says 172.21.15.96:36142) 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2505b640 1 -- 172.21.15.96:0/4121170618 learned_addr learned my addr 172.21.15.96:0/4121170618 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2585c640 1 -- 172.21.15.96:0/4121170618 >> v1:172.21.15.96:6789/0 conn(0x7f5e2006e5c0 legacy=0x7f5e2006ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2485a640 1 -- 172.21.15.96:0/4121170618 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062094519 0 0) 0x7f5e20071670 con 0x7f5e2006e5c0 2024-09-13T17:45:55.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2585c640 1 -- 172.21.15.96:0/4121170618 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200a7200 con 0x7f5e20134670 2024-09-13T17:45:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.130+0000 7f5e2585c640 1 --2- 172.21.15.96:0/4121170618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e20134670 0x7f5e20134a50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000a8e0 tx=0x7f5e1001d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b37735dbe11d756 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2485a640 1 -- 172.21.15.96:0/4121170618 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e100a7600 con 0x7f5e20134670 2024-09-13T17:45:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2485a640 1 -- 172.21.15.96:0/4121170618 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100a7760 con 0x7f5e20134670 2024-09-13T17:45:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2485a640 1 -- 172.21.15.96:0/4121170618 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e100a7a20 con 0x7f5e20134670 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/4121170618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e20134670 msgr2=0x7f5e20134a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/4121170618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e20134670 0x7f5e20134a50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5e1000a8e0 tx=0x7f5e1001d760 comp rx=0 tx=0).stop 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/4121170618 shutdown_connections 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/4121170618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e20134670 0x7f5e20134a50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/4121170618 >> 172.21.15.96:0/4121170618 conn(0x7f5e20098a60 msgr2=0x7f5e2009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/4121170618 shutdown_connections 2024-09-13T17:45:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/4121170618 wait complete. 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 Processor -- start 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- start start 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e200a2370 con 0x7f5e2006e5c0 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2585c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2585c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39128/0 (socket says 172.21.15.96:39128) 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2585c640 1 -- 172.21.15.96:0/1218332623 learned_addr learned my addr 172.21.15.96:0/1218332623 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2585c640 1 -- 172.21.15.96:0/1218332623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1000a5c0 con 0x7f5e2006e5c0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2585c640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001a370 tx=0x7f5e1001bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e100af920 con 0x7f5e2006e5c0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e100afaa0 con 0x7f5e2006e5c0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e200a1390 con 0x7f5e2006e5c0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5e100afd80 con 0x7f5e2006e5c0 2024-09-13T17:45:55.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e201ca3d0 con 0x7f5e2006e5c0 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f5e100b9450 con 0x7f5e2006e5c0 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e2009d520 con 0x7f5e2006e5c0 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 0x7f5e040379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f5e100ab070 con 0x7f5e2006e5c0 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.134+0000 7f5e2505b640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 0x7f5e040379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.138+0000 7f5e2505b640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 0x7f5e040379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007fd0 tx=0x7f5e1c007920 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.143 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.142+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f5e100c2d80 con 0x7f5e2006e5c0 2024-09-13T17:45:55.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.306+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e200a1a80 con 0x7f5e040354e0 2024-09-13T17:45:55.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.306+0000 7f5e167fc640 1 -- 172.21.15.96:0/1218332623 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5e200a1a80 con 0x7f5e040354e0 2024-09-13T17:45:55.310 INFO:teuthology.orchestra.run.smithi096.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 msgr2=0x7f5e040379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 0x7f5e040379a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007fd0 tx=0x7f5e1c007920 comp rx=0 tx=0).stop 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 msgr2=0x7f5e200a3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f5e1001a370 tx=0x7f5e1001bfa0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 shutdown_connections 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f5e040354e0 0x7f5e040379a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 --2- 172.21.15.96:0/1218332623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e2006e5c0 0x7f5e200a3030 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 >> 172.21.15.96:0/1218332623 conn(0x7f5e20098a60 msgr2=0x7f5e2012f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:55.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 shutdown_connections 2024-09-13T17:45:55.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.310+0000 7f5e27ac0640 1 -- 172.21.15.96:0/1218332623 wait complete. 2024-09-13T17:45:55.328 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:45:55.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1904641640 1 Processor -- start 2024-09-13T17:45:55.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1904641640 1 -- start start 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1904641640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f18fc09a810 0x7f18fc13a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1904641640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18fc09c020 con 0x7f18fc098a60 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1904641640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18fc09cc90 con 0x7f18fc09a810 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f19023dd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f18fc098a60 0x7f18fc098e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36150/0 (socket says 172.21.15.96:36150) 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1901bdc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f18fc09a810 0x7f18fc13a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f19023dd640 1 -- 172.21.15.96:0/1930631830 learned_addr learned my addr 172.21.15.96:0/1930631830 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:45:55.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1901bdc640 1 -- 172.21.15.96:0/1930631830 >> v1:172.21.15.96:6789/0 conn(0x7f18fc098a60 legacy=0x7f18fc098e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f1901bdc640 1 -- 172.21.15.96:0/1930631830 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18fc09c2e0 con 0x7f18fc09a810 2024-09-13T17:45:55.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.442+0000 7f19013db640 1 -- 172.21.15.96:0/1930631830 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479483682 0 0) 0x7f18fc09c020 con 0x7f18fc098a60 2024-09-13T17:45:55.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1901bdc640 1 --2- 172.21.15.96:0/1930631830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc09a810 0x7f18fc13a110 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009740 tx=0x7f18ec01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0febb64e3894ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19013db640 1 -- 172.21.15.96:0/1930631830 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec0ab530 con 0x7f18fc09a810 2024-09-13T17:45:55.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19013db640 1 -- 172.21.15.96:0/1930631830 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ec0a7050 con 0x7f18fc09a810 2024-09-13T17:45:55.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19013db640 1 -- 172.21.15.96:0/1930631830 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18ec0ab7e0 con 0x7f18fc09a810 2024-09-13T17:45:55.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/1930631830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc09a810 msgr2=0x7f18fc13a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 --2- 172.21.15.96:0/1930631830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc09a810 0x7f18fc13a110 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f18ec009740 tx=0x7f18ec01c390 comp rx=0 tx=0).stop 2024-09-13T17:45:55.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/1930631830 shutdown_connections 2024-09-13T17:45:55.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 --2- 172.21.15.96:0/1930631830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc09a810 0x7f18fc13a110 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/1930631830 >> 172.21.15.96:0/1930631830 conn(0x7f18fc06e350 msgr2=0x7f18fc070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:55.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/1930631830 shutdown_connections 2024-09-13T17:45:55.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/1930631830 wait complete. 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 Processor -- start 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- start start 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18fc139f10 con 0x7f18fc098a60 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19023dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19023dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39152/0 (socket says 172.21.15.96:39152) 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19023dd640 1 -- 172.21.15.96:0/3560311773 learned_addr learned my addr 172.21.15.96:0/3560311773 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:45:55.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19023dd640 1 -- 172.21.15.96:0/3560311773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec009420 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f19023dd640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c390 tx=0x7f18f800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f8007410 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f8007590 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18fc0a6140 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18f80047f0 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.446+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18fc0a4880 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 38) v1 ==== 43353+0+0 (secure 0 0 0) 0x7f18f8004950 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18fc139990 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f18f2ffd640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 0x7f18e00379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f18f8047350 con 0x7f18fc098a60 2024-09-13T17:45:55.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f1901bdc640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 0x7f18e00379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:45:55.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.450+0000 7f1901bdc640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 0x7f18e00379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0ad000 tx=0x7f18ec008040 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:45:55.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.454+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f18f801bda0 con 0x7f18fc098a60 2024-09-13T17:45:55.531 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:55.530+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:55.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.618+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18fc0a5160 con 0x7f18e00354e0 2024-09-13T17:45:55.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.618+0000 7f18f2ffd640 1 -- 172.21.15.96:0/3560311773 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13864 (secure 0 0 0) 0x7f18fc0a5160 con 0x7f18e00354e0 2024-09-13T17:45:55.621 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:45:55.621 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:45:55.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 msgr2=0x7f18e00379a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 0x7f18e00379a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0ad000 tx=0x7f18ec008040 comp rx=0 tx=0).stop 2024-09-13T17:45:55.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 msgr2=0x7f18fc1cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:45:55.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f18f800c390 tx=0x7f18f800c850 comp rx=0 tx=0).stop 2024-09-13T17:45:55.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 shutdown_connections 2024-09-13T17:45:55.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f18e00354e0 0x7f18e00379a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 --2- 172.21.15.96:0/3560311773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18fc098a60 0x7f18fc1cab80 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:45:55.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 >> 172.21.15.96:0/3560311773 conn(0x7f18fc06e350 msgr2=0x7f18fc135470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:45:55.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 shutdown_connections 2024-09-13T17:45:55.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:45:55.622+0000 7f1904641640 1 -- 172.21.15.96:0/3560311773 wait complete. 2024-09-13T17:45:55.640 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-13T17:45:53.806837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":127,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":88380,"kb_used_data":3224,"kb_used_omap":4,"kb_used_meta":85115,"kb_avail":283027140,"statfs":{"total":289910292480,"available":289819791360,"internally_reserved":0,"allocated":3301376,"data_stored":2024105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":87158099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5135,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":-52,"num_write_kb":-22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.691126"},"pg_stats":[{"pgid":"1.0","version":"74'3500","reported_seq":3763,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-09-13T17:45:51.788003+0000","last_change":"2024-09-13T17:41:52.907530+0000","last_active":"2024-09-13T17:45:51.788003+0000","last_peered":"2024-09-13T17:45:51.788003+0000","last_clean":"2024-09-13T17:45:51.788003+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:45:51.788003+0000","last_undegraded":"2024-09-13T17:45:51.788003+0000","last_fullsized":"2024-09-13T17:45:51.788003+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T17:41:51.881130+0000","last_clean_scrub_stamp":"2024-09-13T17:41:51.881130+0000","objects_scrubbed":0,"log_size":3500,"log_dups_size":0,"ondisk_log_size":3500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T23:15:37.202354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":576,"num_read_kb":1057,"num_write":224,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3500,"ondisk_log_size":3500,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738417,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31796,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":696320,"data_stored":278697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.45200000000000001,"15min":0.442},"min":{"1min":0.30099999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51600000000000001,"15min":0.50800000000000001},"min":{"1min":0.28899999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.74099999999999999,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:44:54 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.45600000000000002,"15min":0.44500000000000001},"min":{"1min":0.25600000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.71599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53300000000000003,"15min":0.52900000000000003},"min":{"1min":0.40400000000000003,"5min":0.161,"15min":0.161},"max":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.66200000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738418,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.42899999999999999,"15min":0.42299999999999999},"min":{"1min":0.26600000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60699999999999998,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.29299999999999998}]},{"osd":2,"last update":"Fri Sep 13 17:44:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28499999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46600000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.33900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28292,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343548,"statfs":{"total":96636764160,"available":96607793152,"internally_reserved":0,"allocated":1302528,"data_stored":872704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45600000000000002,"15min":0.46000000000000002},"min":{"1min":0.247,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.71399999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.47999999999999998,"15min":0.48999999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.66300000000000003,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Sep 13 17:44:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.23799999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.61899999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.27000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68600000000000005,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:45:55.643 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T17:41:51.881130+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=41, tm_sec=51, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=17, tm_min=45, tm_sec=52, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T17:45:55.643 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T17:45:56.608 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:56.606+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:56.732 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:45:57.730 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:57.726+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:45:58.870 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:45:58.866+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:00.031 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:00.030+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:01.210 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:01.206+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:02.380 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:02.378+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:03.555 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:03.550+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:04.746 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:04.742+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:05.945 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:05.942+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:07.152 INFO:tasks.ceph.mgr.x.smithi096.stderr:2024-09-13T17:46:07.150+0000 7fe7f1173640 -1 client.5508: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.5292:fd12c8d2-c12f-41a2-8a41-eef94f08556b:v1:172.21.15.96:0/1018463101] 2024-09-13T17:46:15.644 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T17:46:15.777 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.773+0000 7f2bee089640 1 Processor -- start 2024-09-13T17:46:15.778 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.773+0000 7f2bee089640 1 -- start start 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2be806f6d0 0x7f2be8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be806e130 con 0x7f2be809dc20 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80a6750 con 0x7f2be806f6d0 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be77fe640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f2be806f6d0 0x7f2be8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f2be809dc20 0x7f2be8134ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36406/0 (socket says 172.21.15.96:36406) 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be6ffd640 1 -- 172.21.15.96:0/1756628543 learned_addr learned my addr 172.21.15.96:0/1756628543 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be77fe640 1 -- 172.21.15.96:0/1756628543 >> v1:172.21.15.96:6789/0 conn(0x7f2be809dc20 legacy=0x7f2be8134ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:15.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be77fe640 1 -- 172.21.15.96:0/1756628543 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80712e0 con 0x7f2be806f6d0 2024-09-13T17:46:15.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be77fe640 1 --2- 172.21.15.96:0/1756628543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be8134ab0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009740 tx=0x7f2bd801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5938ab4dcd883a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:15.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be67fc640 1 -- 172.21.15.96:0/1756628543 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bd80ab400 con 0x7f2be806f6d0 2024-09-13T17:46:15.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be67fc640 1 -- 172.21.15.96:0/1756628543 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80a7050 con 0x7f2be806f6d0 2024-09-13T17:46:15.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2be67fc640 1 -- 172.21.15.96:0/1756628543 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bd80ab6b0 con 0x7f2be806f6d0 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- 172.21.15.96:0/1756628543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 msgr2=0x7f2be8134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 --2- 172.21.15.96:0/1756628543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be8134ab0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8009740 tx=0x7f2bd801c390 comp rx=0 tx=0).stop 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- 172.21.15.96:0/1756628543 shutdown_connections 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 --2- 172.21.15.96:0/1756628543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be8134ab0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- 172.21.15.96:0/1756628543 >> 172.21.15.96:0/1756628543 conn(0x7f2be8098a60 msgr2=0x7f2be809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.777+0000 7f2bee089640 1 -- 172.21.15.96:0/1756628543 shutdown_connections 2024-09-13T17:46:15.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- 172.21.15.96:0/1756628543 wait complete. 2024-09-13T17:46:15.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 Processor -- start 2024-09-13T17:46:15.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- start start 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80a1ca0 con 0x7f2be806f6d0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47298/0 (socket says 172.21.15.96:47298) 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be77fe640 1 -- 172.21.15.96:0/1472747595 learned_addr learned my addr 172.21.15.96:0/1472747595 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be77fe640 1 -- 172.21.15.96:0/1472747595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd8009420 con 0x7f2be806f6d0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be77fe640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80ae040 tx=0x7f2bd801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bd80b7450 con 0x7f2be806f6d0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80b75b0 con 0x7f2be806f6d0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be80a2c10 con 0x7f2be806f6d0 2024-09-13T17:46:15.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2bd80b7850 con 0x7f2be806f6d0 2024-09-13T17:46:15.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be80a2f20 con 0x7f2be806f6d0 2024-09-13T17:46:15.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 39) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f2bd80b79b0 con 0x7f2be806f6d0 2024-09-13T17:46:15.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 0x7f2bd4037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:15.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be80a3f10 con 0x7f2be806f6d0 2024-09-13T17:46:15.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f2bd80e21f0 con 0x7f2be806f6d0 2024-09-13T17:46:15.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.781+0000 7f2be6ffd640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 0x7f2bd4037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:15.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.785+0000 7f2be6ffd640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 0x7f2bd4037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000f70 tx=0x7f2bdc007a90 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:15.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.789+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f2bd80c78f0 con 0x7f2be806f6d0 2024-09-13T17:46:15.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.945+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 --> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2be8135910 con 0x7f2bd4035570 2024-09-13T17:46:15.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.945+0000 7f2bcffff640 1 -- 172.21.15.96:0/1472747595 <== mgr.5476 v2:172.21.15.96:6824/3414482076 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13690 (secure 0 0 0) 0x7f2bdc001300 con 0x7f2bd4035570 2024-09-13T17:46:15.950 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:46:15.950 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 msgr2=0x7f2bd4037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 0x7f2bd4037a30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000f70 tx=0x7f2bdc007a90 comp rx=0 tx=0).stop 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 msgr2=0x7f2be80a40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f2bd80ae040 tx=0x7f2bd801abd0 comp rx=0 tx=0).stop 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 shutdown_connections 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f2bd4035570 0x7f2bd4037a30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 --2- 172.21.15.96:0/1472747595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2be806f6d0 0x7f2be80a40d0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:15.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 >> 172.21.15.96:0/1472747595 conn(0x7f2be8098a60 msgr2=0x7f2be8133f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:15.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 shutdown_connections 2024-09-13T17:46:15.953 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:15.949+0000 7f2bee089640 1 -- 172.21.15.96:0/1472747595 wait complete. 2024-09-13T17:46:15.968 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-09-13T17:46:15.812531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":681,"num_read_kb":1274,"num_write":244,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4983,"ondisk_log_size":4983,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":2,"kb":283115520,"kb_used":104800,"kb_used_data":3260,"kb_used_omap":4,"kb_used_meta":101499,"kb_avail":283010720,"statfs":{"total":289910292480,"available":289802977280,"internally_reserved":0,"allocated":3338240,"data_stored":2037524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4781,"internal_metadata":103935315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":208,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002619"},"pg_stats":[{"pgid":"1.0","version":"75'4983","reported_seq":5301,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-13T17:46:08.249092+0000","last_change":"2024-09-13T17:45:55.861971+0000","last_active":"2024-09-13T17:46:08.249092+0000","last_peered":"2024-09-13T17:46:08.249092+0000","last_clean":"2024-09-13T17:46:08.249092+0000","last_became_active":"2024-09-13T17:41:52.905510+0000","last_became_peered":"2024-09-13T17:41:52.905510+0000","last_unstale":"2024-09-13T17:46:08.249092+0000","last_undegraded":"2024-09-13T17:46:08.249092+0000","last_fullsized":"2024-09-13T17:46:08.249092+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"74'3930","last_scrub_stamp":"2024-09-13T17:45:55.861645+0000","last_deep_scrub":"74'3930","last_deep_scrub_stamp":"2024-09-13T17:45:55.861645+0000","last_clean_scrub_stamp":"2024-09-13T17:45:55.861645+0000","objects_scrubbed":2,"log_size":4983,"log_dups_size":0,"ondisk_log_size":4983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-15T02:28:31.898583+0000","scrub_duration":0.014632289,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":681,"num_read_kb":1274,"num_write":244,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":681,"num_read_kb":1274,"num_write":244,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4983,"ondisk_log_size":4983,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738421,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31816,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":716800,"data_stored":285596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45600000000000002,"15min":0.44400000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.73299999999999998,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52300000000000002,"15min":0.51000000000000001},"min":{"1min":0.27200000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.76900000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Sep 13 17:45:57 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46500000000000002,"15min":0.44800000000000001},"min":{"1min":0.25700000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.74199999999999999,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53500000000000003,"15min":0.53000000000000003},"min":{"1min":0.35999999999999999,"5min":0.161,"15min":0.161},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.66100000000000003}]}]},{"osd":1,"up_from":8,"seq":34359738422,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1310720,"data_stored":875964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 17:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.42099999999999999,"15min":0.41999999999999998},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.57999999999999996,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.501},{"interface":"front","average":{"1min":0.374,"5min":0.44,"15min":0.45800000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.57299999999999995,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Sep 13 17:45:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.44800000000000001,"15min":0.45800000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.61099999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.47399999999999998,"15min":0.46500000000000002},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.65800000000000003,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57499999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36492,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335348,"statfs":{"total":96636764160,"available":96599396352,"internally_reserved":0,"allocated":1310720,"data_stored":875964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.44,"15min":0.45400000000000001},"min":{"1min":0.161,"5min":0.161,"15min":0.161},"max":{"1min":0.59199999999999997,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.46100000000000002,"15min":0.48399999999999999},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.60199999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Sep 13 17:45:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45100000000000001,"15min":0.45700000000000002},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.61499999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48099999999999998,"15min":0.496},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T17:46:15.969 DEBUG:teuthology.orchestra.run.smithi096:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T17:46:16.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f69929af640 1 Processor -- start 2024-09-13T17:46:16.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f69929af640 1 -- start start 2024-09-13T17:46:16.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f69929af640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7f698c09dc20 0x7f698c1351c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f69929af640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f698c06e130 con 0x7f698c06f6d0 2024-09-13T17:46:16.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f69929af640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698c0a6750 con 0x7f698c09dc20 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f698bfff640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7f698c06f6d0 0x7f698c134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36418/0 (socket says 172.21.15.96:36418) 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.089+0000 7f698bfff640 1 -- 172.21.15.96:0/2531304952 learned_addr learned my addr 172.21.15.96:0/2531304952 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698b7fe640 1 --2- 172.21.15.96:0/2531304952 >> v2:172.21.15.96:3300/0 conn(0x7f698c09dc20 0x7f698c1351c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698affd640 1 -- 172.21.15.96:0/2531304952 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850463820 0 0) 0x7f698c06e130 con 0x7f698c06f6d0 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698affd640 1 -- 172.21.15.96:0/2531304952 --> v1:172.21.15.96:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6974003530 con 0x7f698c06f6d0 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698b7fe640 1 -- 172.21.15.96:0/2531304952 >> v1:172.21.15.96:6789/0 conn(0x7f698c06f6d0 legacy=0x7f698c134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698b7fe640 1 -- 172.21.15.96:0/2531304952 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0712e0 con 0x7f698c09dc20 2024-09-13T17:46:16.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698b7fe640 1 --2- 172.21.15.96:0/2531304952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c09dc20 0x7f698c1351c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6980009d90 tx=0x7f698001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604e4d93e856922a server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698affd640 1 -- 172.21.15.96:0/2531304952 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69800a5640 con 0x7f698c09dc20 2024-09-13T17:46:16.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698affd640 1 -- 172.21.15.96:0/2531304952 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69800a57a0 con 0x7f698c09dc20 2024-09-13T17:46:16.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f698affd640 1 -- 172.21.15.96:0/2531304952 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69800a5a60 con 0x7f698c09dc20 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 -- 172.21.15.96:0/2531304952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c09dc20 msgr2=0x7f698c1351c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 --2- 172.21.15.96:0/2531304952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c09dc20 0x7f698c1351c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f6980009d90 tx=0x7f698001c8e0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 -- 172.21.15.96:0/2531304952 shutdown_connections 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 --2- 172.21.15.96:0/2531304952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c09dc20 0x7f698c1351c0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 -- 172.21.15.96:0/2531304952 >> 172.21.15.96:0/2531304952 conn(0x7f698c098a60 msgr2=0x7f698c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 -- 172.21.15.96:0/2531304952 shutdown_connections 2024-09-13T17:46:16.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.093+0000 7f69929af640 1 -- 172.21.15.96:0/2531304952 wait complete. 2024-09-13T17:46:16.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 Processor -- start 2024-09-13T17:46:16.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 -- start start 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698c0a1b30 con 0x7f698c06f6d0 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47318/0 (socket says 172.21.15.96:47318) 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698bfff640 1 -- 172.21.15.96:0/997171157 learned_addr learned my addr 172.21.15.96:0/997171157 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698bfff640 1 -- 172.21.15.96:0/997171157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980009a70 con 0x7f698c06f6d0 2024-09-13T17:46:16.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698bfff640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f698400c450 tx=0x7f698400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6984005c20 con 0x7f698c06f6d0 2024-09-13T17:46:16.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6984005d80 con 0x7f698c06f6d0 2024-09-13T17:46:16.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c0a3560 con 0x7f698c06f6d0 2024-09-13T17:46:16.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69840071a0 con 0x7f698c06f6d0 2024-09-13T17:46:16.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f698c132ee0 con 0x7f698c06f6d0 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 39) v1 ==== 43409+0+0 (secure 0 0 0) 0x7f6984007300 con 0x7f698c06f6d0 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f698c134ab0 con 0x7f698c06f6d0 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 0x7f6978037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f6984046b90 con 0x7f698c06f6d0 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698b7fe640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 0x7f6978037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.097+0000 7f698b7fe640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 0x7f6978037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69800af000 tx=0x7f69800a7070 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.105+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7f6984010b80 con 0x7f698c06f6d0 2024-09-13T17:46:16.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.253+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f698c0a1ed0 con 0x7f698c06f6d0 2024-09-13T17:46:16.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.261+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6984010b80 con 0x7f698c06f6d0 2024-09-13T17:46:16.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.261+0000 7f6988ff9640 1 -- 172.21.15.96:0/997171157 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v13)=0 v13) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6984017030 con 0x7f698c06f6d0 2024-09-13T17:46:16.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 msgr2=0x7f6978037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 0x7f6978037a30 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f69800af000 tx=0x7f69800a7070 comp rx=0 tx=0).stop 2024-09-13T17:46:16.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 msgr2=0x7f698c0a07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f698400c450 tx=0x7f698400c910 comp rx=0 tx=0).stop 2024-09-13T17:46:16.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 shutdown_connections 2024-09-13T17:46:16.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7f6978035570 0x7f6978037a30 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 --2- 172.21.15.96:0/997171157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f698c06f6d0 0x7f698c0a07c0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 >> 172.21.15.96:0/997171157 conn(0x7f698c098a60 msgr2=0x7f698c131c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:16.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 shutdown_connections 2024-09-13T17:46:16.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.265+0000 7f69929af640 1 -- 172.21.15.96:0/997171157 wait complete. 2024-09-13T17:46:16.283 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-13T17:46:16.284 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-13T17:46:16.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.397+0000 7fdbdd2be640 1 Processor -- start 2024-09-13T17:46:16.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.397+0000 7fdbdd2be640 1 -- start start 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbd8137490 0x7fdbd8139880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- --> v1:172.21.15.96:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbd8071670 con 0x7fdbd8134670 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- --> v2:172.21.15.96:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd8071360 con 0x7fdbd8137490 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd67fc640 1 --2- >> v2:172.21.15.96:3300/0 conn(0x7fdbd8137490 0x7fdbd8139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd6ffd640 1 --1- >> v1:172.21.15.96:6789/0 conn(0x7fdbd8134670 0x7fdbd8134a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.96:6789/0 says I am v1:172.21.15.96:36434/0 (socket says 172.21.15.96:36434) 2024-09-13T17:46:16.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd6ffd640 1 -- 172.21.15.96:0/1027302803 learned_addr learned my addr 172.21.15.96:0/1027302803 (peer_addr_for_me v1:172.21.15.96:0/0) 2024-09-13T17:46:16.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd67fc640 1 -- 172.21.15.96:0/1027302803 >> v1:172.21.15.96:6789/0 conn(0x7fdbd8134670 legacy=0x7fdbd8134a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd5ffb640 1 -- 172.21.15.96:0/1027302803 <== mon.0 v1:172.21.15.96:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305249462 0 0) 0x7fdbd8071670 con 0x7fdbd8134670 2024-09-13T17:46:16.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd67fc640 1 -- 172.21.15.96:0/1027302803 --> v2:172.21.15.96:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd80a7200 con 0x7fdbd8137490 2024-09-13T17:46:16.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd67fc640 1 --2- 172.21.15.96:0/1027302803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8137490 0x7fdbd8139880 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009740 tx=0x7fdbcc01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f1213b87b8672d server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd5ffb640 1 -- 172.21.15.96:0/1027302803 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc0ab530 con 0x7fdbd8137490 2024-09-13T17:46:16.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd5ffb640 1 -- 172.21.15.96:0/1027302803 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdbcc0a7050 con 0x7fdbd8137490 2024-09-13T17:46:16.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbd5ffb640 1 -- 172.21.15.96:0/1027302803 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbcc0ab800 con 0x7fdbd8137490 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1027302803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8137490 msgr2=0x7fdbd8139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1027302803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8137490 0x7fdbd8139880 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc009740 tx=0x7fdbcc01c390 comp rx=0 tx=0).stop 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1027302803 shutdown_connections 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1027302803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8137490 0x7fdbd8139880 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1027302803 >> 172.21.15.96:0/1027302803 conn(0x7fdbd8098a60 msgr2=0x7fdbd809ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1027302803 shutdown_connections 2024-09-13T17:46:16.406 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.401+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1027302803 wait complete. 2024-09-13T17:46:16.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 Processor -- start 2024-09-13T17:46:16.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 -- start start 2024-09-13T17:46:16.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd809f790 con 0x7fdbd8134670 2024-09-13T17:46:16.407 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47330/0 (socket says 172.21.15.96:47330) 2024-09-13T17:46:16.408 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd6ffd640 1 -- 172.21.15.96:0/1877332862 learned_addr learned my addr 172.21.15.96:0/1877332862 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd6ffd640 1 -- 172.21.15.96:0/1877332862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc009420 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd6ffd640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c420 tx=0x7fdbc800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc80074d0 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fdbc8007630 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdbc8004980 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd80a67d0 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd809e510 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 39) v1 ==== 43409+0+0 (secure 0 0 0) 0x7fdbc8004ae0 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd8134a50 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 0x7fdbc4037a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 3017+0+0 (secure 0 0 0) 0x7fdbc8046c90 con 0x7fdbd8134670 2024-09-13T17:46:16.410 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.405+0000 7fdbd67fc640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 0x7fdbc4037a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-13T17:46:16.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.409+0000 7fdbd67fc640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 0x7fdbc4037a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ad000 tx=0x7fdbcc0a6000 comp rx=0 tx=0).ready entity=mgr.5476 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-13T17:46:16.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.413+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175519 (secure 0 0 0) 0x7fdbc804d080 con 0x7fdbd8134670 2024-09-13T17:46:16.591 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.589+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbd809f540 con 0x7fdbd8134670 2024-09-13T17:46:16.592 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.589+0000 7fdbbf7fe640 1 -- 172.21.15.96:0/1877332862 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2015 (secure 0 0 0) 0x7fdbc801ab40 con 0x7fdbd8134670 2024-09-13T17:46:16.592 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-13T17:46:16.592 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":45,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5370,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6827/3475774537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3475774537},{"type":"v1","addr":"172.21.15.96:6827","nonce":3475774537}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":38},{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6829/3246074756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3246074756},{"type":"v1","addr":"172.21.15.96:6829","nonce":3246074756}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":45}],"filesystems":[]} 2024-09-13T17:46:16.592 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 45 2024-09-13T17:46:16.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 msgr2=0x7fdbc4037a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 0x7fdbc4037a30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0ad000 tx=0x7fdbcc0a6000 comp rx=0 tx=0).stop 2024-09-13T17:46:16.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 msgr2=0x7fdbd813a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-13T17:46:16.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fdbc800c420 tx=0x7fdbc800c8e0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.594 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 shutdown_connections 2024-09-13T17:46:16.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:6824/3414482076,v1:172.21.15.96:6825/3414482076] conn(0x7fdbc4035570 0x7fdbc4037a30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 --2- 172.21.15.96:0/1877332862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbd8134670 0x7fdbd813a0e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-13T17:46:16.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 >> 172.21.15.96:0/1877332862 conn(0x7fdbd8098a60 msgr2=0x7fdbd812de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-13T17:46:16.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 shutdown_connections 2024-09-13T17:46:16.595 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-13T17:46:16.593+0000 7fdbdd2be640 1 -- 172.21.15.96:0/1877332862 wait complete. 2024-09-13T17:46:16.609 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-13T17:46:16.609 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T17:46:16.610 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-13T17:46:16.610 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:16.620 INFO:tasks.ceph.mds.a:Stopped 2024-09-13T17:46:16.620 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-13T17:46:16.620 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:16.670 INFO:tasks.ceph.mds.b:Stopped 2024-09-13T17:46:16.671 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T17:46:16.671 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T17:46:16.671 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:21.528 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-13T17:46:22.772 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T17:46:22.773 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T17:46:22.773 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:22.854 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T17:46:22.854 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T17:46:22.854 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:22.937 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T17:46:22.938 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T17:46:22.938 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T17:46:22.938 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:22.971 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T17:46:22.972 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T17:46:22.972 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T17:46:22.972 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T17:46:23.033 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T17:46:23.033 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T17:46:23.033 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-13T17:46:23.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:46:23.089 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T17:46:23.646 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:46:23.646 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T17:46:23.839 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:46:23.839 DEBUG:teuthology.orchestra.run.smithi096:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T17:46:23.996 INFO:tasks.ceph:Archiving mon data... 2024-09-13T17:46:23.998 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909/data/mon.a.tgz 2024-09-13T17:46:23.998 DEBUG:teuthology.orchestra.run.smithi096:> mktemp 2024-09-13T17:46:24.004 INFO:teuthology.orchestra.run.smithi096.stdout:/tmp/tmp.e1oOsoawB1 2024-09-13T17:46:24.004 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.e1oOsoawB1 2024-09-13T17:46:24.251 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0666 /tmp/tmp.e1oOsoawB1 2024-09-13T17:46:24.315 DEBUG:teuthology.orchestra.remote:smithi096:/tmp/tmp.e1oOsoawB1 is 1MB 2024-09-13T17:46:24.384 DEBUG:teuthology.orchestra.run.smithi096:> rm -fr /tmp/tmp.e1oOsoawB1 2024-09-13T17:46:24.392 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T17:46:24.392 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T17:46:24.447 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T17:46:24.447 DEBUG:teuthology.orchestra.run.smithi096:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T17:46:24.508 INFO:teuthology.orchestra.run.smithi096.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T17:46:24.512 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T17:46:24.514 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909/remote/smithi096/crash 2024-09-13T17:46:24.515 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T17:46:24.563 INFO:tasks.ceph:Shutting down logrotate 2024-09-13T17:46:24.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-13T17:46:24.643 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-13T17:46:24.698 INFO:tasks.ceph:Compressing logs... 2024-09-13T17:46:24.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:46:26.529 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-13T17:46:28.068 INFO:tasks.ceph:Archiving logs... 2024-09-13T17:46:28.068 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909/remote/smithi096/log 2024-09-13T17:46:28.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T17:46:28.738 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T17:46:28.759 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T17:46:28.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T17:46:28.804 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T17:46:28.812 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T17:46:28.813 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout: remote refid st t when poll reach delay offset jitter 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:============================================================================== 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:+hv01.front.sepi 67.205.162.81 3 u 48 64 177 0.069 -1.333 1.193 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:+hv02.front.sepi 63.231.80.2 3 u 45 64 177 0.086 -3.151 1.163 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout:*hv03.front.sepi 74.6.168.72 3 u 49 64 177 0.113 +0.489 1.788 2024-09-13T17:46:28.829 INFO:teuthology.orchestra.run.smithi096.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-13T17:46:28.830 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T17:46:28.839 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T17:46:28.839 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T17:46:28.847 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T17:46:28.855 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T17:46:28.863 INFO:teuthology.task.internal:Duration was 900.768301 seconds 2024-09-13T17:46:28.863 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T17:46:28.871 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T17:46:28.871 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T17:46:28.908 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T17:46:28.908 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-13T17:46:28.909 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T17:46:28.958 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T17:46:28.959 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T17:46:29.033 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T17:46:29.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T17:46:29.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T17:46:29.163 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T17:46:29.163 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T17:46:29.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T17:46:29.184 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:46:29.228 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-13T17:46:29.240 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T17:46:29.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T17:46:29.290 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T17:46:29.300 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T17:46:29.300 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-13_14:42:16-fs-wip-rishabh-testing-20240910.100730-reef-distro-default-smithi/7903909/remote/smithi096 2024-09-13T17:46:29.301 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T17:46:29.356 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T17:46:29.356 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T17:46:29.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T17:46:29.408 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T17:46:29.408 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T17:46:29.419 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T17:46:29.419 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T17:46:29.447 INFO:teuthology.orchestra.run.smithi096.stdout: 658617 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 17:46 /home/ubuntu/cephtest 2024-09-13T17:46:29.448 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T17:46:29.458 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T17:46:29.479 INFO:teuthology.run:Summary data: description: fs/top/{begin/{0-install 1-ceph 2-logrotate 3-modules} cluster/{1-node} conf/{client mds mgr mon osd} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health pg_health} supported-random-distros$/{ubuntu_latest} tasks/fstop} duration: 900.7683012485504 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-09-13T17:46:29.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T17:46:29.571 INFO:teuthology.run:pass